What are flags used for in TensorFlow

What is the difference between tf.placeholder and tf.Variable?


I'm a newbie to TensorFlow. I am confused about the difference between and. From my point of view it is used for input data and to save the data status. That is everything I know.

Could someone explain their differences to me in more detail? In particular, when and when to use?



Reply:


In short, you are using trainable variables like weights (W) and distortions (B) for your model.

is used to feed current training examples.

How to feed the training examples during training:

You will be trained (modified) as a result of this training.

For more information, see https://www.tensorflow.org/versions/r0.7/tutorials/mnist/tf/index.html. (Examples are from the website.)





The difference is that you have to specify an initial value when you declare it. With you don't have to provide an initial value and you specify it at run time with the arguments inside






Since tensor calculations are made up of graphs, it is better to interpret the two in terms of graphs.

Take simple linear regression, for example

where and stand for the weights and distortions and for the inputs of the observations and for the outputs of the observations.

Obvious and are of the same nature (manifest variables) different from those of and (latent variables). and are values ​​of the samples (observations) and therefore require a space to be filled , while and are the weights and the deviation variables (the previous values ​​affect the latter) in the graph that should be trained using different and pairs. We place different samples in the Placeholders , to the variables to train .

We just have to variables (at checkpoints) save or restore to save or recreate the diagram with the code.

placeholder are mostly owners of the various data sets (e.g. training data or test data). However, in the training process variables trained for the specific tasks, ie to predict the result of the input or to assign the input to the desired labels. They stay the same until you recreate the model with different or the same examples work out or optimize , to the placeholder often by the dictation to fill . For example:

placeholder are also used as parameters for Set from Models handed over.

If you change the placeholders (add, delete, change shape, etc.) of a model during training, you can still reload the checkpoint without further changes. However, if the variables of a saved model are changed, you should adjust the checkpoint accordingly in order to reload it and continue training (all variables defined on the graph should be available in the checkpoint).

In conclusion, if the values ​​are from the samples (observations that you already have), you are sure to create a placeholder to hold them. If you need a parameter to train, use one variable (put simply, make the variables automatically use a TF for the desired values).

In some interesting models, such as a style transfer model, the input pixels are optimized and the variables usually referred to as model variables are set. Then we should make the input (usually randomly initialized) as a variable as implemented in this link.

Please refer to this simple and illustrative document for more information.


TL; DR

variables

  • So that you learn parameters
  • Values ​​can be derived from training
  • Initial values ​​are required (often random)

placeholder

  • Allocated memory for data (e.g. for image pixel data during a feed)
  • Initial values ​​are not required (but can be set, see)

The most obvious difference between the tf.Variable and the tf.placeholder is the following


You use variables to store and update parameters. Variables are in-memory buffers that contain tensors. They must be explicitly initialized and can be saved on the hard drive during and after training. You can restore saved values ​​later to train or analyze the model.

The variables are initialized with. When creating a variable, you must also pass a tensor as an initial value to the constructor. When you create a variable, you always know its shape.


On the other hand, you cannot update the placeholder. They shouldn't be initialized either, but since they're a promise to a tensor, you need to feed the value into them. Finally, when compared to a variable, the placeholder may not know the shape. You can either specify parts of the dimensions or nothing at all.


There are other differences:

It is interesting that not only placeholders can be fed. You can pass the value to a variable and even a constant.


Aside from others' answers, they also explain this very well in this MNIST tutorial on the Tensoflow website:

We describe these interaction operations by manipulating symbolic variables. Let's create one:

,

is not a definite value. It's a placeholder, a value that we enter when we ask TensorFlow to do a calculation. We want to be able to input any number of MNIST images, each flattened into a 784 dimensional vector. We represent this as a 2-D tensor of floating point numbers with a form [None, 784]. (Here none means that a dimension can be arbitrarily long.)

We also need the weights and prejudices for our model. We could think of treating these like additional inputs, but TensorFlow has an even better way of dealing with it:. A is a modifiable tensor included in TensorFlow's diagram of interaction operations. It can be used and even modified by the calculation. For machine learning applications, in general, the model parameters must be s.

We create these s by giving the initial value of:. In this case we initialize both and as tensors full of zeros. Because we are going to learn and it doesn't matter very much what they are in the beginning.




Tensorflow uses three types of containers to save / run the process

  1. Constants: Constants contain the typical data.

  2. Variables: Data values ​​are changed with the respective functions such as cost_function.

  3. Placeholder: Training / test data are transferred to the graphic.


Sample excerpt:

As the name suggests, a placeholder is a promise to provide a value later, i. H.

variables are simply the training parameters ((Matrix), (Bias) which correspond to the normal variables you use in your daily programming and which the trainer updates / changes with each run / step.

While placeholder does not require an initial value that when you created and allocated no memory to TF, rather than later when you use the placeholders in the charge, TensorFlow will allocate the appropriately sized memory for them (and) - this unconstrained- With this function we can store data of any size and enter the shape.


In a nutshell :

variable - is a parameter that the trainer (ie GradientDescentOptimizer) should update after each step.

Placeholder- Demo -

Execution:

which leads to the output

In the first case, 3 and 4.5 are passed to or or to adder_node ouputting 7. In the second case there is a feed list. The first steps 1 and 2 are added, the next 3 and 4 (and).


Relevant reads:


variables

A TensorFlow variable is the best way to represent a common, permanent state that has been manipulated by your program. Variables are processed using the tf.Variable class. Internally, a tf.variable stores a persistent tensor. With certain operations you can read and change the values ​​of this tensor. These changes are visible in several tf.Sessions, so that several workers can see the same values ​​for a tf.Variable. Variables must be initialized before they can be used.

Example:

This creates a calculation diagram. The variables (x and y) can be initialized as follows and the function (f) can be evaluated in a Tensorflow session:

placeholder

A placeholder is a node (like a variable) whose value can be initialized in the future. These nodes basically output the value assigned to them at runtime. A placeholder node can be assigned using the tf.placeholder () class, for which you can specify arguments such as the type of variable and / or its shape. Wildcards are often used to represent the training data set in a machine learning model because the training data set is constantly changing.

Example:

Note: "None" for a dimension means "any size".

References:

  1. https://www.tensorflow.org/guide/variables
  2. https://www.tensorflow.org/api_docs/python/tf/placeholder
  3. O'Reilly: Hands-on machine learning with Scikit-Learn & Tensorflow

In Tensorflow, imagine normal variables that we use in programming languages. We initialize variables, we can also change them later. No initial value is required. The placeholder simply allocates a block of memory for future use. We can feed in the data later. By default, it has an unrestricted shape that allows you to feed tensors of different shape in one session. You can create a constrained shape by passing the optional -shap argument shape, as described below.

During the machine learning task, most of the time we don't know the number of rows, but (let's say) we know the number of features or columns. In this case we can use None.

Now we can feed each matrix with 4 columns and any number of rows at runtime.

Wildcards are also used for input data (these are variables that we feed our model with), where variables are parameters like weights that we train over time.


Placeholder:

  1. A placeholder is simply a variable that we will assign data to at a later date. It allows us to build our operations and build our calculation chart without needing the data. In TensorFlow terminology, we then enter data into the chart using these placeholders.

  2. Initial values ​​are not required, but can have default values ​​with

  3. We need to provide a value at runtime like:

Variable:

  1. A TensorFlow variable is the best way to represent a common, permanent state that has been manipulated by your program.
  2. Variables are processed via the class tf.Variable. A tf.Variable represents a tensor whose value can be changed by executing ops.

Example:


Tensorflow 2.0 compatible answer : The concept of wildcards is not available by default as the default execution mode is eager execution.

However, we can use them if they are used in ().

Corresponding command for TF Placeholder in version 2.x is.

The equivalent command for the TF variable in version 2.x is. If you want to migrate the code from 1.x to 2.x, the equivalent command is

.

More information about Tensorflow version 2.0 can be found on this Tensorflow page.

For more information on migrating from version 1.x to version 2.x, see the migration guide.


Introduce yourself Calculation diagram before . In such a diagram we need an input node to feed our data to the diagram to hand over . These Nodes should be used as placeholders in the Tensorflow can be defined .

Don't think of it as a general program in Python. You can write a Python program and do all the things that other answers just explained by variables. For calculation diagrams in Tensorflow, however, you have to define these nodes as placeholders in order to add your data to the diagram.

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from.

By continuing, you consent to our use of cookies and other tracking technologies and affirm you're at least 16 years old or have consent from a parent or guardian.

You can read details in our Cookie policy and Privacy policy.