ni / grpc-device

gRPC server providing remote access to NI device driver APIs.

NI gRPC Device Server and Client APIs

This repo contains the source code needed to build and run the NI gRPC Device Server for supported NI hardware driver APIs. Also contained in this repo are instructions and examples that demonstrate how to create client code that interacts with devices connected to an instance of the NI gRPC Device Server.

The server and the client APIs allow NI's instrumentation to be accessed and controlled through a remote interface via pre-defined APIs using a client/server architecture. The API is not a driver but instead a layer on top of the existing driver C APIs that provides remote capabilities.

For more detailed information on the server and API design refer to the wiki.

Supported NI drivers

Indicates the most recent driver version used to test builds of the current source. Supported driver versions for specific releases will be found in the release notes for that version.

NI Driver	Version Tested (Windows)	Version Tested (Linux)	Version Tested (Linux RT)
NI-DAQmx	2023 Q1	2023 Q1	2023 Q1
NI-DCPower	2023 Q1	2023 Q1	2023 Q1
NI-Digital Pattern Driver	2023 Q1	Not Supported	Not Supported
NI-DMM	2023 Q1	2023 Q1	2023 Q1
NI-FGEN	2023 Q1	2023 Q1	2023 Q1
NI-RFmx Bluetooth	2024 Q1	Not Supported	Not Supported
NI-RFmx CDMA2k	2023 Q1	Not Supported	Not Supported
NI-RFmx Demod	2023 Q1	Not Supported	Not Supported
NI-RFmx GSM	2023 Q1	Not Supported	Not Supported
NI-RFmx LTE	2024 Q1	Not Supported	Not Supported
NI-RFmx NR	2024 Q1	Not Supported	Not Supported
NI-RFmx SpecAn	2024 Q1	Not Supported	Not Supported
NI-RFmx TD-SCDMA	2023 Q1	Not Supported	Not Supported
NI-RFmx WCDMA	2023 Q1	Not Supported	Not Supported
NI-RFmx WLAN	2024 Q1	Not Supported	Not Supported
NI-RFSA	21.0.0	21.0.0	Not Supported
NI-RFSG	21.0.0	21.0.0	Not Supported
NI-SCOPE	2023 Q2	2023 Q2	2023 Q2
NI-SWITCH	2023 Q1	2023 Q1	2023 Q1
NI-TClk	2023 Q1	2023 Q1	2023 Q1
NI-VISA	2024 Q1	Not Supported	Not Supported
NI-XNET	21.5.0	21.5.0	21.5.0

Build Status

Downloading a Release

Download the Server

1. (Windows Only) Download and install the latest Microsoft Visual C++ Redistributable for Visual Studio 2017 and 2019.
2. Navigate to the Releases page.
3. Download the latest Server Release .tar.gz or .zip for the desired platform.
4. Extract the contents of the .tar.gz or .zip to a directory with read and write permissions.
5. Run the server

• List of supported OS:
  • Windows 64-bit
  • Linux 64-bit
  • NI Linux RT

Download the Client Files

1. Navigate to the Releases page.
2. Download the latest Client Release's ni-grpc-device-client.tar.gz or ni-grpc-device-client.zip depending on the client platform.
3. Extract the contents of ni-grpc-device-client.tar.gz or ni-grpc-device-client.zip to a directory with read and write permissions.
4. Create a gRPC client.

Building Locally

If you're looking to build the grpc-device repo locally, look at the Getting Started section of CONTRIBUTING.md.

Running the gRPC Server

The server's startup configuration is set by specifying port and security settings in a JSON configuration file. A default configuration file named server_config.json with an insecure configuration (no SSL/TLS) bound to localhost is located in the same directory as the server executable. For more information on SSL/TLS related security settings refer to the SSL/TLS Support section. For more information on address binding refer to the Bind Address Support section. The location of the server binary is not important as long as the user has proper permissions in the chosen directory.

There are two ways to start the server:

1. Launch the server application without specifying a path to a configuration file (use the default configuration file):

   Windows

   .\ni_grpc_device_server.exe

   Note: It is also possible to start the server by double-clicking the executable. Starting the server through a command prompt, however, allows for observation of startup errors.

   Linux and Linux RT

   ./ni_grpc_device_server

2. Launch the server application by specifying a path (relative or absolute) to the configuration file:

   Windows

   .\ni_grpc_device_server.exe C:\path\to\config\file\server_config.json

   Linux and Linux RT

   ./ni_grpc_device_server /path/to/config/file/server_config.json

If the server starts successfully on the port specified in the configuration file, then it will print a message to the terminal output:

Server listening on port 12345. Security is configured with insecure credentials.

Note: If port 0 is specified then the server will automatically select a port from the dynamic range. The port used will be reflected in the startup message.

If the server fails to start (i.e. a port is not specified in the configuration file) then an error message is printed in the terminal and the application will exit.

Common Server Startup Errors

1. The datatypes of the values in the configuration file don't match the expected datatypes. For example, the port must be an integer type and not a string. The error message will provide specific details on the type requirements.
2. The configuration file can't be found at the provided location. This error can also occur if the user lacks read permissions for the file.
3. The server configuration file is malformed and is not in proper JSON format. Refer to the JSON configuration file in this readme for an example of the expected format.
4. The specified address is not valid. The solution is to select a valid IPv4 or IPv6 address.
5. The specified port is out of the allowed port range. The solution is to select a port in the allowable range (0-65535).
6. The specified port is already in use. The solution is to select another port or terminate the other application using the port.
7. Security configuration errors. See Server Security Support wiki page.

Default Configuration File (localhost):

Below are the contents of a default configuration file accepting localhost connections on port 31763 and configured without SSL/TLS. A configuration file with these contents also exists in the same directory as the ni_grpc_device_server binary.

{
    "address": "[::1]",
    "port": 31763,
    "security" : {
       "server_cert": "",
       "server_key": "",
       "root_cert": ""
    }
 }

Bind Address Support

The server supports specifying the address to bind to. The address can be used to enable local or remote connections. Address values include any valid IPv4 or IPv6 address. To bind to local (loopback) connection, specify address "[::1]". To bind to any address, specify address "[::]". If no address is specified, the server configuration defaults to any address "[::]".

Licensing behaviour

If you are using gRPC to control a licensed software, e.g. RFmx, the license checkout will happen on the machine running the gRPC server, not in the client side. As the license check-in happens, when the process which requested the license terminates, you need to close the gRPC server application to return the license.

Creating a gRPC Client

Each supported driver API has a corresponding .proto file that defines the interface used by clients to interact with the NI devices connected to the server. Creating a client requires compiling the .proto into supporting files in the client's language of choice using the protocol buffer compiler protoc. For more detailed information refer to the Creating a gRPC Client wiki page.

SSL/TLS Support

The server supports both server-side TLS and mutual TLS. Security configuration is accomplished by setting the server_cert, server_key and root_cert values in the server's configuration file. The server expects the certificate files specified in the configuration file to exist in a certs folder that is located in the same directory as the configuration file being used by the server. For more detailed information on SSL/TLS support refer to the Server Security Support wiki page.

grpc-device's Issues

A couple methods in niscope_service.custom.cpp aren't checking status correctly.

A few of the custom scope methods are setting the response fields only if the status is not zero.
This should be reversed, as a zero status represents a successful API call.

For instance, GetNormalizationCoefficients currently has this:

if (status != 0) {
      response->set_number_of_coefficient_sets(number_of_coefficient_sets);
      Copy(coefficient_info, response->mutable_coefficient_info());
    }

When it should be keying off of status == 0 like the other methods.

RFSG and SpecAn GetError tests fail (order dependent)

ErrorFromDriver_GetErrorMessage_ReturnsUserErrorMessage

Both of these tests can fail depending on what tests they run with.

Both of these are passing GetError a session id from a request that failed to init the session. So there's some weirdness in how we're handling that.

AB#1873996

Server SSL TLS Support wiki page link in readme is broken

The link to Server SSL TLS Support wiki page in the main readme.md is broken. It looks like the wiki page was renamed.

LICENSE and NOTICE files are missing from the release archives.

For license compliance, and to facilitate downstream packaging (in NI LinuxRT), license and notice files from the grpc-device repo and any applicable subcomponents should be included in all independent export artifacts.

At a minimum, the ni-grpc-device-server-ni-linux-rt-x64.tar.gz artifact here should include all legal notices.

Attribute enum values starting with S get mangled by C# protoc generator

Reported by @smooresni

This happens because our attribute prefix (NIRFSA_ATTRIBUTE) does not match the attribute name (NiRfsaAttributes). Normally this would defeat the nice-to-have prefix stripping in the C# client generator. But when the value starts with S the prefix-stripping works and takes the S off with it.

AB#1728728

TIMING_ATTRIBUTE_SAMP_QUANT_SAMP_PER_CHAN has the wrong datatype

It's marked as a double attribute, but DAQmxGetSampQuantSampPerChan is a u64.

It looks like this is caused because we're using lvDataType from the metadata to determine the datatype. In this case, the declared lv type is inconsistent with the C/internal datatype.

We need to fix this specific issue and analyze the metadata for similar inconsistencies.

AB#1593158

Python client code generation example for betterproto outputs unnecessary files

On the https://github.com/ni/grpc-device/wiki/Creating-a-gRPC-Client wiki page, it shows an example for generating betterproto client code for python:

b. Better Protobuf compiler:

> mkdir nidevice
> cd nidevice
> python -m grpc_tools.protoc -I="..\..\..\proto" --python_betterproto_out=. --grpc_python_out=. session.proto
> python -m grpc_tools.protoc -I="..\..\..\proto" --python_betterproto_out=. --grpc_python_out=. niscope.proto

However, this unnecessarily includes the standard compiler GRPC files with the --grpc_python_out flag (in this case, outputting session_pb2_grpc.py and niscope_pb2_grpc.py), which aren't needed/used with betterproto. The example can be simplified to:

mkdir nidevice
cd nidevice
python -m grpc_tools.protoc -I="..\..\..\proto" --python_betterproto_out=. session.proto
python -m grpc_tools.protoc -I="..\..\..\proto" --python_betterproto_out=. niscope.proto

P.S. Really excited about this grpc-device project! At last, able to easily connect to multiple NI devices over the network from any client, without painful driver installations. Thanks for all your work on this!

Update Linux RT Feed to point to new release 1.5.2

Linux RT needs to be updated to point to the new grpc-device release.

Update the following lines in recipes-ni/ni-grpc-device/ni-grpc-device_git.bb in ni/meta-nilrt

• PV = "1.5.2"
• SRCREV_grpc-device = "1e91e023bc03bae25b52cebc52a1a12c3ce720c8"

AB#2063544

Update Linux RT Feed to point to new release 1.5.2

Linux RT needs to be updated to point to the new grpc-device release.

Update the following lines in recipes-ni/ni-grpc-device/ni-grpc-device_git.bb in ni/meta-nilrt

• PV = "1.5.2"
• SRCREV_grpc-device = "1e91e023bc03bae25b52cebc52a1a12c3ce720c8"

AB#2063517

Any plans on supporting Unix Domain Sockets?

Are there any plans on supporting unix domain sockets. These are available for both windows and linux and will assit with IPC on the same machine.

AB#1762814

Add supported RFmx versions to readme.md

Server version 1.4 adds support for RFmx. Can we add supported version numbers to this table in the readme?

grpc-device does not fill in response fields when a warning is returned

Our standard response handling looks like this:

  response->set_status(status);
  if (status == 0) {
    response->set_x0_out(x0_out);
    response->set_dx_out(dx_out);
    convert_to_grpc(waveform_out, response->mutable_waveform_out());
    response->set_actual_array_size(actual_array_size);
    response->set_papr(papr);
    response->set_power_offset(power_offset);
  }
  return ::grpc::Status::OK;

In RFmx (and maybe other drivers) you don't typically want to abort on a warning (warning -> status > 0, unless it's an ivi-dance param) and can use the output values. We should check the standard error handling macros and other higher-level wrappers for drivers and update the ones that continue on warnings.

AB#1762292

RFSG attribute methods use inconsistent parameter name

Specifically, here's a sample request:

message SetAttributeViBooleanRequest {
  nidevice_grpc.Session vi = 1;
  string channel_name = 2;
  NiRFSGAttributes attribute = 3;
  bool value = 4;
}

but RFSA and most other drivers use attribute_id instead of attribute.

AB#1724759

Calling nidaqmx GetExtendedErrorInfo via gRPC is not reliable because it uses thread-local storage

nidaqmx.proto supports the DAQmxGetExtendedErrorInfo function. Calling this function via gRPC is not reliable because it uses thread-local storage.

Depending which server thread is used to handle this request, it may return the correct error message, an empty string, or an incorrect error message.

Steps to reproduce:

1. Change one of the DAQmx examples to make multiple concurrent GetExtendedErrorInfo requests:

def raise_if_error(response):
    """Raise an exception if an error was returned."""
    if response.status != 0:
        response = client.GetExtendedErrorInfo.future(nidaqmx_types.GetExtendedErrorInfoRequest())
        response2 = client.GetExtendedErrorInfo.future(nidaqmx_types.GetExtendedErrorInfoRequest())
        response3 = client.GetExtendedErrorInfo.future(nidaqmx_types.GetExtendedErrorInfoRequest())
        raise Exception(f"Error: {response3.result().error_string}")

2. Run NI IO Trace and enable the ThreadID column.
3. Run the example with an invalid device identifier. Normally, this would display a "Device identifier is invalid" error, but this issue may cause it to display an empty string or an incorrect error message.

(.venv) PS Z:\grpc-device\examples\nidaqmx> python .\analog-input.py localhost 31763 Dev42/ai0
Traceback (most recent call last):
  File ".\analog-input.py", line 71, in <module>
    raise_if_error(
  File ".\analog-input.py", line 63, in raise_if_error
    raise Exception(f"Error: {response3.result().error_string}")
Exception: Error: The requested memory could not be allocated.
Status Code: -50352

NI IO Trace will show which thread was used to handle each call to DAQmxGetExtendedErrorInfo:

AB#2085088

DPDApplyDigitalPredistortionSplitRequest has output parameters that should be in the response

This is a gap in the metadata, possibly because the higher-level APIs combine the waveform_out into one field.

message DPDApplyDigitalPredistortionSplitRequest {
  nidevice_grpc.Session instrument = 1;
  string selector_string = 2;
  double x0_in = 3;
  double dx_in = 4;
  repeated float waveform_in_i = 5;
  repeated float waveform_in_q = 6;
  bool idle_duration_present = 7;
  double measurement_timeout = 8;
  repeated float waveform_out_i = 9;
  repeated float waveform_out_q = 10;
  int32 array_size_out = 11;
}

AB#1759890

Improve documentation of security considerations/exposure

From NITech presentation.

Those docs look good, but I think it really need to be emphasized that any authenticated client is essentially granted root access to the machine (depending on driver + hardware).

AB#1561894

RFSG LO Source range table enums have an extra "T" in the name

gRPC Server's IVI-Dance has race conditions

See code in https://github.com/ni/grpc-device/blob/main/generated/nidcpower/nidcpower_service.cpp#L2400

In IVI based APIs, the way clients retrieve strings is by calling the function to get the size, allocating, then calling again with the actual buffer (aka "the IVI Dance").

It is being done incorrectly: the recursive session lock isn't grabbed, so it is possible for the size of the string to grow between the time in which the size is queried and the time in which the string is copied onto the client buffer.

There are two ways to fix it:

1. Acquire the session lock. This is how NI's own C# and Python APIs do it internally.
2. Keep growing the buffer in a loop until it's big enough, which is usually just the one time. This is how NI's own Digital Pattern Editor does it internaly.

AB#1579918

SpecAn enable_all_traces is an int32

Seems like this should be a bool. Double-check C# implementation.

message SelectMeasurementsRequest {
  nidevice_grpc.Session instrument = 1;
  string selector_string = 2;
  oneof measurements_enum {
    MeasurementTypes measurements = 3;
    uint32 measurements_raw = 4;
  }
  int32 enable_all_traces = 5;
}

AB#1759901

grpc-device exports do not include NOTICE files for subprojects which are a part of built binaries.

Some subcomponents of grpc-device are licensed under MIT (json); others under Apache-2 (grpc). To ensure license compliance, the grpc-device repo (and any built artifacts from the repo), should include notices that built binaries might contain code or derivatives from those projects.

An easy way to do this might be to append the LICENSE file with a notice about the applicable subprojects and their licenses. Otherwise, independent NOTICE files might be appropriate. Any notice content should be included in all export archives - per issue #180.

Update Linux RT Feed to point to new release 1.5.2

Linux RT needs to be updated to point to the new grpc-device release.

Update the following lines in recipes-ni/ni-grpc-device/ni-grpc-device_git.bb in ni/meta-nilrt

• PV = "1.5.2"
• SRCREV_grpc-device = "1e91e023bc03bae25b52cebc52a1a12c3ce720c8"

AB#2063503

DAQ's GetErrorString() should remove trailing NULL characters in the error_string

Since we know this parameter is a string, we should remove trailing NULLs for this and all strings in the API.

AB#1560643

grpc-device should convert string encoding to/from UTF-8 when necessary

Protocol Buffers requires strings to be encoded as UTF-8. However, many of the NI driver APIs supported by grpc-device use the Windows current ANSI code page, which is typically a single-byte encoding like ISO-8859-1 or a multi-byte encoding like Shift JIS.

NI driver APIs typically support internationalization for:

• User-defined names, such as device aliases, DAQmx global channel names, and DAQmx task names
• File paths
• Localized error messages
• Serialized configuration data for import/export

I haven't tested internationalization with grpc-device, but I would expect these results:

• Sending strings that are not structurally valid UTF-8 may result in errors or coerced string contents.
• Sending or receiving strings in languages that natively support Unicode (like C# and Python3) may result in mojibake.

Driver APIs:

• DAQmx: nicaiu.dll uses ANSI/MBCS. nicai_utf8.dll uses UTF-8, but requires a different import library than the one that is checked into imports\lib\win64.
• MI, RF, XNET I think these all use ANSI/MBCS.
  • Exception: The MI ImportAttributeConfigurationBuffer and ExportAttributeConfigurationBuffer functions pass an array of bytes containing a UTF-8 JSON string. Related issue: #694

AB#2108036

grpc-device should not expose LockSession/UnlockSession

These lock methods have a thread affinity, but we're executing everything on a thread pool. So you're acquiring a lock that you can't use unless the subsequent access and release happen to be scheduled on the same thread.

LockSession/UnlockSession are already released for the MI drivers but they don't work, so it should be easy to remove them.

The grpc-device Reserve/Unreserve methods should be used instead.

Server allocated session id should be initialized to 1

Ethernet frame is missing last four bytes in the payload

Using XNET WriteFrame API, I am writing an ethernet frame. After reading the same frame using ReadFrame API, the last four bytes of payload are zeroed out.

Possible performance issue

I am testing the gRPC server to use for a project and running into some performance issues..
Running the following code results in ReadAnalogF64 method progressively taking longer to execute

using var channel = Grpc.Net.Client.GrpcChannel.ForAddress(@"http://localhost:31763");
NationalInstruments.Grpc.Device.SessionUtilities.SessionUtilitiesClient client = new NationalInstruments.Grpc.Device.SessionUtilities.SessionUtilitiesClient(channel);

var devices = client.EnumerateDevices(new NationalInstruments.Grpc.Device.EnumerateDevicesRequest()).Devices;
foreach (var device in devices)
{
	Console.WriteLine($"{device.Name} - Model: {device.Model} - Serial: {device.SerialNumber}");
}

NiDAQmx.NiDAQmxClient daqMxClient = new NiDAQmx.NiDAQmxClient(channel);
var createTaskResp = daqMxClient.CreateTask(new CreateTaskRequest() { SessionName = "BAI_TEST" });

var createAIChanResp = daqMxClient.CreateAIVoltageChan(new CreateAIVoltageChanRequest()
{
	Task = createTaskResp.Task,
	PhysicalChannel = $"cDAQ9188-1718E7EMod8/AI0:8",
	NameToAssignToChannel = "AI",
	TerminalConfig = InputTermCfgWithDefault.CfgDefault,
	MinVal = -10,
	MaxVal = 10,
	Units = VoltageUnits2.Volts
});

var CfgSampClkTimingResp = daqMxClient.CfgSampClkTiming(new CfgSampClkTimingRequest() { Task = createTaskResp.Task, Rate = 1000, SampleMode = AcquisitionType.FiniteSamps, ActiveEdge = Edge1.Rising, SampsPerChan = 500 });

daqMxClient.StartTask(new StartTaskRequest() { Task = createTaskResp.Task });

var sp = new Stopwatch();

for (int i = 0; i < 100; i++)
{
	sp.Start();
	daqMxClient.ReadAnalogF64(new ReadAnalogF64Request() { Task = createTaskResp.Task, NumSampsPerChan = 5, Timeout = 10, FillMode = GroupBy.GroupByChannel, ArraySizeInSamps = 10 * 5 });//.Dump("Read");
	sp.Elapsed.Dump();
	sp.Stop();
}
Thread.Sleep(100);
daqMxClient.StopTask(new StopTaskRequest() { Task = createTaskResp.Task });
daqMxClient.ClearTask(new ClearTaskRequest() { Task = createTaskResp.Task });

Result

cDAQ9188-1718E7EMod6 - Model: NI 9425 - Serial: 015930FC
cDAQ9188-1718E7EMod2 - Model: NI 9401 - Serial: 01596BC3
cDAQ9188-1718E7EMod5 - Model: NI 9425 - Serial: 015930FC
cDAQ9188-1718E7EMod4 - Model: NI 9263 - Serial: 017160A2
cDAQ9188-1718E7EMod8 - Model: NI 9205 (DSUB) - Serial: 015A2A60
cDAQ9188-1718E7E - Model: NI cDAQ-9188 - Serial: 01718E7E
00:00:00.0155992
00:00:00.0319618
00:00:00.0328618
00:00:00.0335054
00:00:00.0466991
00:00:00.0471030
00:00:00.0475674
00:00:00.0625285
00:00:00.0631418
00:00:00.0636510
00:00:00.0778650
00:00:00.0783266
00:00:00.0786581
00:00:00.0935619
00:00:00.0941926
00:00:00.0946180
00:00:00.1086067
00:00:00.1092140
00:00:00.1096643
00:00:00.1100289
00:00:00.1240900
00:00:00.1244472
00:00:00.1246849
00:00:00.1401118
00:00:00.1405757
00:00:00.1409394
00:00:00.1553003
00:00:00.1558009
00:00:00.1563109
00:00:00.1710808
00:00:00.1715601
00:00:00.1718873
00:00:00.1871156
00:00:00.1875493
00:00:00.1878285
00:00:00.2019579
00:00:00.2025948
00:00:00.2030137
00:00:00.2181133
00:00:00.2186977
00:00:00.2190665
00:00:00.2329346
00:00:00.2337302
00:00:00.2341432
00:00:00.2491135
00:00:00.2497718
00:00:00.2503696
00:00:00.2646442
00:00:00.2649295
00:00:00.2651653
00:00:00.2653436
00:00:00.2811086
00:00:00.2817649
00:00:00.2823658
00:00:00.2966592
00:00:00.2971803
00:00:00.2977992
00:00:00.3181848
00:00:00.3187701
00:00:00.3195061
00:00:00.3285954
00:00:00.3293039
00:00:00.3300340
00:00:00.3304685
00:00:00.3446340
00:00:00.3453890
00:00:00.3458263
00:00:00.3595224
00:00:00.3600098
00:00:00.3603269
00:00:00.3753621
00:00:00.3758440
00:00:00.3763261
00:00:00.3915148
00:00:00.3919496
00:00:00.3923501
00:00:00.4079287
00:00:00.4085615
00:00:00.4090118
00:00:00.4232711
00:00:00.4240436
00:00:00.4245194
00:00:00.4385397
00:00:00.4391226
00:00:00.4396426
00:00:00.4401134
00:00:00.4544155
00:00:00.4549320
00:00:00.4552887
00:00:00.4691281
00:00:00.4695173
00:00:00.4699073
00:00:00.4853418
00:00:00.4860935
00:00:00.4867072
00:00:00.5013470
00:00:00.5020301
00:00:00.5024541
00:00:00.5028741
00:00:00.5033567

Create releases notes tracking tag and template

We need a way to mark PRs for inclusion in release notes for any breaking changes or important bug fixes.

Before closing this, #379 needs to be tagged for inclusion in the release notes OR we need to revert the API breakage.

Python grpc client examples require Python 3.6 or newer

Currently, the Python examples take advantage of f-strings. That functionality was not added until Python 3.6 as per PEP 498. This can cause issues if the examples are run with a NI Linux Real-Time target as the client-side since those devices currently only include Python 2.7 and Python 3.5.5 (as of the creation of this issue).

dB in param names is mis-converted in rfmx specan

Example: oned_b_compression_point

AB#1759875

nidcpower and nifgen have incorrect data type for Import/ExportAttributeConfigurationBuffer

In the MI ImportAttributeConfigurationBuffer and ExportAttributeConfigurationBuffer functions, the configuration parameter is an array of bytes. The bytes contain a UTF-8 JSON string, but the API documentation doesn't document this, so it's appropriate for grpc-device to treat this parameter as an array of bytes.

nidcpower.proto and nifgen.proto incorrectly represent this parameter as repeated fixed64 configuration.

nidmm.proto, nifake.proto, and niscope.proto correctly represent this parameter as bytes configuration.

AB#2108035

README misspells products

NI-DCPOWER is spelled NI-DCPower
NI-DIGITAL PATTERN DRIVER is spelled NI-Digital Pattern Driver
NI-TCLK is spelled NI-TClk

There may be other instances across the repository. All the NI product names with their correct spelling can be found in ni.com.

AB#1736521

Coerced scalar inputs do not implement range checking

This is implemented for arrays and structs but not for scalars. This affects int16, uInt16, and int8 params. All of these are represented as 32-bit types in protobuf. If a number is passed in that doesn't fit in the driver type range, it will be truncated.

AB#1759778

Several functions in RFmxSpecAn have "rb_w" parameters

Specifically the functions are:

• ACPCfgRBWFilter
• CCDFCfgRBWFilter
• CHPCfgRBWFilter
• FCntCfgRBWFilter
• HarmCfgFundamentalRBW
• HarmFetchHarmonicMeasurement
• HarmFetchHarmonicMeasurementArray
• IMCfgRBWFilter
• OBWCfgRBWFilter
• SEMCfgCarrierRBWFilter
• SEMCfgOffsetRBWFilter
• SEMCfgOffsetRBWFilterArray
• SpectrumCfgRBWFilter
• SpurCfgRangeRBWArray
• SpurCfgRangeRBWFilter
• TXPCfgRBWFilter

All of these functions have a parameter in their metadata named rbW which gRPC translates to rb_w. We should rename this to rbw in the metadata (and probably check enums and attributes as well)

AB#1751435

Dummy issue to test creation of bug in AzDO automatically

A workflow was recently added to automatically create a bug on AzDO board when an issue is raised on the github repo. Creating this bug to verify the workflow works as expected.

AB#1558833

Enum aliases cause problems in json replay style use cases

Avoiding enum aliases is considered a best practice because the JSON representation has to just "pick one". This issue is described here.

We have some enums where the C API has a zero value but we also generate an UNSPECIFIED zero value. These are marked as allow_alias to allow for that duplicate. When you do a json trace of a gRPC session (i.e., in grpc-dump or with an interceptor), you often get UNSPECIFIED when the actual sent value was something else.

Unfortunately, using UNSPECIFIED for the zero value of an enum is also a best practice. So, these 2 conventions are in conflict, assuming that we want to preserve the mapping to the C API.

We have been discussing using gRPC json based APIs for SCPI style workflows, which would include trace/copy/paste. The aliases break that workflow because you end up capturing bogus "UNSPECIFIED" values. (OR does this work but just look weird?)

We need some research to consider our options:

1. Can we remove the UNSPECIFIED field when there is a zero value in the C API? What is the effect on backwards compatibility (assuming users aren't intentionally sending UNSPECIFIED)?
2. Can we use "mapped" enums in this case? So that the enum values aren't necessarily the same as the C API values.
3. Are "raw" values an acceptable alternative? Or does this work "well enough"?

AB#1758247

RFmxSpecan BuildIntermodStringRequest has outputs in request

message BuildIntermodStringRequest {
  string selector_string = 1;
  int32 intermod_number = 2;
  int32 selector_string_out_length = 3;
  string selector_string_out = 4;
}

message BuildIntermodStringResponse {
  int32 status = 1;
}

Registering a DAQmx callback after a task has started crashes the gRPC server and prevents future connection

In testing out various things related to #398, I discovered this unfortunate bug. If you attempt to register a DAQmx callback after the task has already been started, the gRPC server crashes.

I've attached two example python scripts (using betterproto, though it seemingly shouldn't matter what the client is). One script which works daqmx_register_working.py (registering the callback happens before starting the task) and one which causes the crash described above daqmx_register_crash.py (the task is started and underway when the registration is attempted). The files have txt extensions since GitHub doesn't allow for uploading .py files.

The key part / difference is the logic near the bottom. This excerpt is from the crash version:

        print("Starting task")
        await raise_if_error(await daq_service.start_task(task=task))

        print("Sleeping")
        await asyncio.sleep(2)

        print("Trying to register callbacks")
        # THIS WILL FAIL AND CAUSE THE GRPC SERVER TO CRASH
        acquisition = asyncio.gather(read_data(), wait_for_done())
        await acquisition

When running the "crash" script, the client will eventually see:

Starting task
Sleeping
Trying to register callbacks
ERROR: Connection lost
...traceback...

And the gRPC server will no longer be running (seemingly having crashed). Further attempts to connect to the server will fail until it's restarted manually. We didn't see any logs or messaging on the gRPC server side. We're running the server executable on Windows 10.

Perhaps registering a callback after a task has started is not expected to work, but it seems the server should return an error to the client and avoid crashing, since this takes down the full gRPC functionality and requires human intervention to restart.

Thanks in advance for any help/attention. Really appreciate the work on the NI gRPC functionality!

AB#1756795

analog-input-every-n-samples.py can cancel acquisition before reading all samples

If you turn the SampClk rate up high enough (15000-20000 for me) the read_data loop often gets cancelled before acquiring the last batch of samples.

I was hoping that grpc queueing would be able to preserve ordering:

• Events fired in order
• messages enqueued on server in order
• messages sent in order
• messages received and enqueued in order
• messages not necessarily processed in order BUT cancel enqueued on same queue as above
• data processed before cancel

But somewhere that's not working.

AB#1754028

SpecAn StringAttributeValues are mis-named SpecAnCharAttributeValues

AB#1759788

RFmxSpecAn "instrument_handle" parameters should be "instrument"

We think "instrument_handle" is too long, and this is similar to what we did for DAQ ("taskHandle" -> "task")

AB#1751447

SpecAn SpurCfgRangeAbsoluteLimit has optional array parameters that are not supported by grpc-device

C API docs: https://zone.ni.com/reference/en-XX/help/374264N-01/rfmxspecancvi/cvirfmxspecan_spurcfgrangeabso

We don't implement the convention of "Specify NULL if you do not want to set a value for this array", so all arrays must be specified (empty will report an error for mismatched sizes).

I think we can address this b introducing the concept of an optional field and making sure that our codegen supports passing it as null.

AB#1761383

imports\lib\win64 contains LabWindows/CVI user protection libraries and these don't work with /DELAYLOAD

Some of the import libraries in imports\lib\win64 appear to be LabWindows/CVI user protection libraries, not MSVC import libraries:

PS C:\dev\grpc-device> grep -r _UP_ imports\lib\win64
Binary file imports\lib\win64/ivi.lib matches
Binary file imports\lib\win64/nidcpower.lib matches
Binary file imports\lib\win64/niScope.lib matches
Binary file imports\lib\win64/niswitch.lib matches

These are static libraries that contain their own implementation of delay loading as well as LabWindows/CVI-specific user protection functions. However, CMakeLists.txt specifies /DELAYLOAD:nidcpower.dll /DELAYLOAD:niScope.dll /DELAYLOAD:niswitch.dll, which only works with MSVC import libraries.

I think this is why the build process prints warnings about ignored /DELAYLOAD parameters:

LINK : warning LNK4199: /DELAYLOAD:nidcpower.dll ignored; no imports found from nidcpower.dll [C:\Dev\grpc-device\build\ni_grpc_device_server.vcxproj]
LINK : warning LNK4199: /DELAYLOAD:niScope.dll ignored; no imports found from niScope.dll [C:\Dev\grpc-device\build\ni_grpc_device_server.vcxproj]
LINK : warning LNK4199: /DELAYLOAD:niswitch.dll ignored; no imports found from niswitch.dll [C:\Dev\grpc-device\build\ni_grpc_device_server.vcxproj]

I also think this is why these DLLs don't show up in Dependency Walker or https://github.com/lucasg/Dependencies:

(BTW, the correct DLL names on 64-bit Windows are nidcpower_64.dll, niScope_64.dll, and niswitch_64.dll.)

AB#2061280

AB#2082942

Output parameters that are ivi-dance-with-a-twist may be left too big after second call

This is similar to the race condition we found in #247, but sort of the opposite. If the first time we call a function and ask for the parameter's size, resize the parameter, then on the second call the size of the parameter is now smaller, we don't correctly resize the output buffer to the correct size.

We do 0-init these output buffers so there won't be garbage data in them, at least.

@harsha-bhushan - this should be a pretty straightforward fix, do you want it in the next release?

AnalogStatesWithOutputType methods are missing from grpc-device

daqmxgetanalogpowerupstateswithoutputtype
daqmxsetanalogpowerupstateswithoutputtype

Current and voltage power up states can be set through the non-WithOutputType APIs. This missing functionality is the ability to set the PowerUpState to HighImpedance.

AB#1705855

Change RFmxSpecAn TRUE/FALSE attributes back to bools

The RFmx team gave us feedback that using TRUE/FALSE enums is intentional and they want to keep that to preserve the ability to add fields to it. The C# wrapper API also preserves these enums, so it's not just a C-ism.

AB#1761791

ivi-dance-with-a-twist mishandles empty sizes

If the first call to ivi-dance-with-a-twist sets the bufferSize to zero, we should just return without calling again. Currently we call with a zero size, which reports an error (at least for some drivers/APIs).

AB#1707853

betterproto exception error handling does not work in some configurations

See #280

betterproto will raise the underlying socket error instead of a grpc exception. Our examples attempt to filter this for better error reporting, but this does not work consistently. We should apply the DAQ implementation to the other betterproto examples.

AB#1560637

Misnamed parameters in RFmxSpecAn and RFmxInstr

duts_parameters_frequency -> dut_sparameters_frequency or dut_s_parameters_frequency

s_parameter_* -> CONSIDER sparameter_* (matches RFSA/G, but both are reasonable).

s2_p_file_path -> s2p_file_path

seconds_since1970 -> CONSIDER seconds_since_1970

NIRFMXSPECAN_INT32_IMIF_OUTPUT_POWER_OFFSET_AUTO_FALSE -> NIRFMXSPECAN_INT32_IM_IF_OUTPUT_POWER_OFFSET_AUTO_FALSE

NIRFMXSPECAN_INT32_NFY_FACTOR_MODE_MEASURE -> NIRFMXSPECAN_INT32_NF_Y_FACTOR_MODE_MEASURE

pre_cfrpapr -> pre_cfr_papr

PERSONALITY_CDMA2_K -> PERSONALITY_CDMA2K

AB#1790788

Session name conflicts between drivers are reported with error code 1, which is ambiguous

If the session name passed to a session creation RPC (like nidaqmx CreateTask or RF/MI Initialize RPCs) is used by a different driver, the grpc-device server returns error code 1 as a driver API error. However, looking up this error code using the driver API's error message function causes the example programs to displays "Warning: 1" or an inappropriate error message like "Command requires GPIB Controller to be Controller-In-Charge."

Steps to reproduce:

1. Change examples/nidcpower/measure-record.py to set SESSION_NAME = "my task"
2. Run examples/nidcpower/measure-record.py, then while it is running, run examples/nidaqmx/analog-input.py:

(.venv) PS Z:\grpc-device\examples\nidaqmx> python .\analog-input.py
Traceback (most recent call last):
  File ".\analog-input.py", line 77, in <module>
    raise_if_error(response)
  File ".\analog-input.py", line 63, in raise_if_error
    raise Exception(f"Error: {response.error_string}")
Exception: Error: Command requires GPIB Controller to be Controller-In-Charge.

3. Restart the grpc-device server.
4. Add time.sleep(60.0) to the nidaqmx example.
5. Run the nidaqmx example, then while it is running, run the nidcpower example:

(.venv) PS Z:\grpc-device\examples\nidcpower> python .\measure-record.py
Warning: 1
Traceback (most recent call last):
  File ".\measure-record.py", line 107, in <module>
    check_for_error(vi, configure_measure_when.status)
  File ".\measure-record.py", line 68, in check_for_error
    raise Exception(error_message_response.error_message)
Exception: IVI: The session handle is not valid.

It didn't create a NI-DCPower session, but the example program kept going because it reported error code 1 as a warning.

AB#2085853