how to limit GPU usage in this api? about hello_tf_c_api

[Xonxt]

Sorry for reopening this, but I was just struggling with it, and found a somewhat 'automated' solution.

@Neargye @hajungong007

To limit GPU usage in Python, I would write the following.

config = tf.ConfigProto(allow_soft_placement=True)
config.gpu_options.allow_growth=True
config.gpu_options.per_process_gpu_memory_fraction = 0.1

We can now serialize this, and see what Tensorflow expects to get. In my case I get:

serialized = config.SerializeToString()
c = list(map(hex, serialized))
> ['0x32', '0xb', '0x9', '0x9a', '0x99', '0x99', '0x99', '0x99', '0x99', '0xb9', '0x3f', '0x20', '0x1', '0x38', '0x1']

After reverse-engineering this for a bit, I get the following: we need a byte-array of 15 elements, the first three are '0x32', '0xb', '0x9', and the last four are '0x20', '0x1', '0x38', '0x1' (probably corresponding to those two Trues), and between them is just a double value, represented as a byte array (8-bytes).

So, I wrote a simple function:

TF_SessionOptions* CreateSessionOptions ( double percentage ) {
  TF_Status* status = TF_NewStatus();
  TF_SessionOptions* options = TF_NewSessionOptions();

  // the following is an equivalent of setting this in Python:
  // config = tf.ConfigProto( allow_soft_placement = True )
  // config.gpu_options.allow_growth = True
  // config.gpu_options.per_process_gpu_memory_fraction = percentage

  // create a byte-array for the serialized ProtoConfig, set the mandatory bytes (first three and last four)
  uint8_t config[15] = { 0x32, 0xb, 0x9, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x20, 0x1, 0x38, 0x1 };

  // convert the desired percentage into a byte-array
  uint8_t* bytes = reinterpret_cast<uint8_t*>(&percentage);

  // put it to the config byte-array, from 3 to 10:
  for ( int i = 0; i < sizeof( percentage ); i++ ) {     
   config[i + 3] = bytes[i];
  }

  TF_SetConfig( options, (void *) config, 15, status );

  if ( TF_GetCode( status ) != TF_OK ) {
    std::cerr << "Can't set options: " << TF_Message( status ) << std::endl;

    TF_DeleteStatus( status );
    return nullptr;
  }
    
  TF_DeleteStatus( status );
  return options;
}

And use it:

...
TF_Session* session = TF_NewSession( graph, CreateSessionOptions( 0.3 ), status );

I confirm that it works as intended.

[drakkan]

you can do this way

TF_Status* status = TF_NewStatus();
TF_SessionOptions* options = TF_NewSessionOptions();
TF_SetConfig(options,sessionConfigData,sessionConfigSize,status);

where sessionConfigData is serialized as protobuf

see here for more details

tensorflow/tensorflow#13853

[Neargye]

@Xonxt Thanks, it looks very useful.

[hajungong007]

@hajungong007 I got some questions, do you maybe have time to hlep？[email protected]

It can limit GPU usage in this way.

 // GPU =70%
 uint8_t config[11] = {0x32, 0x09, 0x09, 0x66, 0x66, 0x66, 0x66, 0x66, 0x66, 0xe6, 0x3f};
 TF_SetConfig(options.get(), (void*)config, 11, status.get());

or

// GPU = 75%
 uint8_t config[11] = {0x32, 0x09, 0x09, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xe8, 0x3f};
 TF_SetConfig(options.get(), (void*)config, 11, status.get());

or

// GPU = 80%
 uint8_t config[11] = {0x32, 0x09, 0x09, 0x9a, 0x99, 0x99, 0x99, 0x99, 0x99, 0xe9, 0x3f};
 TF_SetConfig(options.get(), (void*)config, 11, status.get());

or

// GPU = 60%
 uint8_t config[11] = {0x32, 0x09, 0x09, 0x33, 0x33, 0x33, 0x33, 0x33, 0x33, 0xe3, 0x3f};
 TF_SetConfig(options.get(), (void*)config, 11, status.get());

[hajungong007]

@ I do not really what you mean here.

I am sorry, it's my code.

[fisakhan]

Using this API how can I confine the TensorFlow to generate one and only one thread irrespective of the number of cores. Currently it is generating N threads (N = number of cores) by setting intra_op_parallelism_threads = 1 and inter_op_parallelism_threads = 1.