Menu

Data Science Dev

Design, develop, deploy

Multithreading TensorFlow / Keras Models

I made an example (with some code borrowed from the Web) of how you can multithread TensorFlow models in Python.  The key I found was to wrap any execution of a model predict function in a lock.  Here is a link to the files.  This is not for training of different models in parallel--only for after you have models already trained and want to use them in parallel threads.

To use follow these instructions:

1. Unzip files.

2. Open terminal:

    a. execute 'python mnist-tf1py' to create CNN model1

    b. execute 'python mnist-tf2.py' to create CNN model2

    c. change paths to models generated by steps 2a and 2b in ts.py

    d. execute 'python ts.py'

    e. execute 'python tc.py model1' or 'python tc.py model2' in another terminal.  I like to type 'python tc.py model1 &' and run it several times.

Go Back

https://stackoverflow.com/questions/40850089/is-keras-thread-safe

From the link above, I think keras is thread safe, so we don't need to add lock when use predict.

Reply


Comment

Blog Search

Blog Archive

Comments