Releases: yahoo/TensorFlowOnSpark
Releases · yahoo/TensorFlowOnSpark
v2.0.0
- initial release compatible with TensorFlow 2.x.
- API changes:
- removed
TFNode.start_cluster_server
, which is not required fortf.keras
andtf.estimator
. - removed
TFNode.export_saved_model
, which can be replaced by TF native APIs now. - added
TFNodeContext.num_workers
to countmaster
,chief
, andworker
nodes.
- removed
- Spark ML Pipeline API changes:
- Scala API has been removed for now, since the Java library for TensorFlow 2.0 is not available yet.
- removed
InputMode.TENSORFLOW
support for ML Pipelines, since the input data is always a Spark DataFrame for this API. - added
HasMasterNode
andHasGraceSecs
params. - removed optional
export_fn
argument for Spark MLTFEstimator
(use TF export APIs instead). - added standard default values for
signature_def_key
andtag_set
for Spark MLTFModel
. - modified inferencing code in
TFModel
for TF2.x APIs.
- older TF 1.x examples have been replaced with TF 2.x compatible examples.
v1.4.4
- last expected release compatible with TensorFlow 1.x (aside from any critical fixes), since the
master
branch will be moving to TF 2.0 compatibility. - handle multiple outputs with signaturedef (thanks to @markromedia).
- handle exceptions after data feeding.
- moved API docs to sphinx_rtd_theme.
- updated to Spark 2.4.4.
v1.4.3
- removed
tensorflow
as a dependency, in order to support other variants liketensorflow-gpu
ortf-nightly
. - allow use of
evaluator
node type in cluster (thanks to @bbshetty) - refactored cluster template generation.
- updated wide-deep example to use models/official code.
- restore termination of feed in mnist/spark example.
- updated sample notebook instructions.
- updated to use Spark 2.3.3.
v1.4.2
v1.4.1
- Added
util.single_node_env()
, which can be used to initialize the environment (HDFS compatibility + GPU allocation) for running a single-node instance of TensorFlow on the Spark driver. - Added an example of parallelized inferencing from a pre-trained SavedModel.
v1.4.0
- More deterministic GPU allocation for multi-GPU nodes.
- Added
timeout
argument toTFCluster.shutdown()
(default is 3 days). This is intended to shutdown the Spark application in the event that any of the TF nodes hang for any reason. Set to -1 to disable timeout. - Added ability to start reservation server on a specific port (contributed by @AvihayTsayeg).
- Updated pipeline API for latest TF APIs (contributed by @AvihayTsayeg)
- Added unit test for
tf.SparseTensor
support. - Updated examples to latest TF APIs (including workaround for tensorflow/tensorflow#21745).
- Updated Spark version dependency for Scala Inferencing API.
- Added
__version__
to module.
v1.3.4
v1.3.3
- Only set TF_CONFIG environment variable if cluster_spec has a "master", i.e. when using
tf.estimator
. - Updated
mnist/keras/mnist_mlp_estimator.py
with example of distributed/parallel inferencing viaestimator.predict
. - Added optional
feed_timeout
argument toTFCluster.train()
for InputMode.SPARK. - Added optional
grace_secs
argument toTFCluster.shutdown()
. - Workaround for firewall proxy issue with
get_ip_address
(contributed by @viplav). - Add support for all Hadoop-compatible File System schemes (contributed by @vishnu2kmohan).
- Added error messages to
assert
statements. - Initial Travis CI integration.
v1.3.2
- add grace period to
TFCluster.shutdown()
- add wide & deep example (contributed by @crafet)
- update mnist/pipeline examples to
tf.data
, add instructions, and misc code cleanup (from @yileic) - parameterize versions in pom.xml and code cleanup (from @tmielika)
- update Scala Inferencing pom.xml to latest tensorflow-hadoop artifact (contributed by @psuszyns)