Note: phiên bản Tiếng Việt của bài này ở link dưới.
https://duongnt.com/ml-net-vie
In 2020, I was in charge of a deep learning project using C#. The model itself was trained in Python with TensorFlow/Keras, but the application to use that model was written in C# .NET Framework. Back then, I used the TensorFlowSharp library to run the model, which was converted into a frozen graph (.pb format). That library worked quite well and served its purpose.
However, things in the AI/ML world move very fast. TensorFlow has gone up to version 2.5.0 at the time of writing, but the TensorFlowSharp library is still stuck at TensorFlow 1.15.0 and appears to be dead. The jump from TensorFlow v1 to v2 brings a lot of performance and functional improvements, but also creates many compatibility issues. Soon, we won’t be able to convert models trained with TensorFlow 2.x into a format that TensorFlowSharp can utilize.
I also looked into the Tensorflow.NET library, and used the knowledge gained to create an age/gender predictor, which you can find here. However, I still felt that something was missing.
Because of that, I switched to ML.NET, a framework created by Microsoft and is now maintained by the .NET Foundation. This framework has some big advantages.
- All versions of TensorFlow up to the latest one are supported.
- It uses the onnx format, which is an open and widely adopted standard. We can use the tf2onnx tool to easily convert frozen graphs, TensorFlow checkpoints, and Keras models into onnx format.
- It is built upon .NET Core/.NET Standard and can run on multiple platforms.
In this article, we will use ML.NET to run a model trained with Keras (TensorFlow backend) and then use it to make some classifications.
The test model
Our test model is a simple CNN network to make classifications on the FashionMNIST dataset, which you can download from here. This model was trained for only 20 epochs and reached just 92% accuracy, but it’s good enough for demonstration purposes. You can download the code to train the model from here. The architecture is below.
We can see that our input node is conv2d_input and our output node is dense_1. Those names will be used when running our model. If necessary, we can use the Netron app to visualize our model and find the input/output name.
The FashionMNIST is what we call a toy dataset, one small enough that we can use it to train models relatively quickly, yet complex enough that reaching 95%+ accuracy is not trivial. It contains 60,000 28×28 grayscale images of 10 different clothing categories. For example, this is a pullover shirt.
Load model and make classification with ML.NET
Load model
First, we need to install both ML.NET and the onnx runtime.
dotnet add package Microsoft.ML --version 1.6.0
dotnet add package Microsoft.ML.OnnxRuntime --version 1.8.1
dotnet add package Microsoft.ML.OnnxTransformer --version 1.6.0
Since our test data in stored as Numpy npz file, we also need to install NumSharp.
dotnet add package NumSharp --version 0.30.0
We map the input/output of our model to these two classes in C#.
public class Input
{
[VectorType(28*28*1)] // Our model takes a three-dimensional tensor as input but ML.NET takes a flatten vector as input
[ColumnName("conv2d_input")] // This name must match the input node's name
public float[] Data { get; set; }
}
public class Output
{
[VectorType(10)] // Because output node has (10) shape
[ColumnName("dense_1")] // This name must match the output node's name
public float[] Data { get; set; }
}
Keep in mind that the input and output of ML.NET is always a one-dimensional vector regardless of the shape of our model’s input/output. For example, our model has input with (28, 28, 1) shape, but the corresponding property in Input class has size 784 == 28 * 28 * 1.
Next, we create a new context and use it to create a pipeline to load our model. Notice that the input and output names are also passed along as two string arrays, one for input and one for output (we need to use arrays even if there is only one input/output).
using Microsoft.ML;
var modelPath = "fashionmnist_cnn_model.onnx";
var outputColumnNames = new[] { "dense_1" };
var inputColumnNames = new[] { "conv2d_input" };
var mlContext = new MLContext();
var pipeline = mlContext.Transforms.ApplyOnnxModel(outputColumnNames, inputColumnNames, modelPath);
Make classification
We will try to classify the sample picture in the previous section, whose npz file you can download from here. The code to load input data and store it in an Input object is below.
var content = np.Load_Npz<byte[,]>("test_image.npz");
var data = np.array(content["test_image.npy"])
.astype(NPTypeCode.Single)
.flatten() // remember to flatten the tensor
.ToArray<float>();
var input = new Input { Data = data };
We can see that the tensor we want to classify is flattened into a one-dimensional array of type float and passed into the Data property of Input object. Then we use this code to make classifications using our pipeline.
var dataView = mlContext.Data.LoadFromEnumerable(input);
var transformedValues = pipeline.Fit(dataView).Transform(dataView);
var output = mlContext.Data.CreateEnumerable<Output>(transformedValues, reuseRowObject: false);
You should receive the following output.
output.Single().Data
// [0, 0, 1, 0, 0, 0, 0, 0, 0, 0]
The value at index==2 has the highest value, which means our model has classified the test image to the category with label 2. From this link, you can see that label 2 is category Pullover.
More advance scenarios
Convert Keras model into onnx format
First, we need to convert the model into saved model format.
from tensorflow.keras.models import saved_model, load_model
h5_path = 'my_model.h5'
saved_model_path = 'my_model' # this is a folder, not a file
model = load_model(h5_path)
save_model(model, saved_model_path)
The code above will create a new folder called my_model which contains the following files and sub-folders.
.
└── assets
└── variables
└── variables.data-00000-of-00001
└── variables.index
└── keras_metadata.pb
└── saved_model.pb
Then we can use tf2onnx to convert from saved_model to onnx. Run the following commands in a terminal window.
pip install tf2onnx
python -m tf2onnx.convert --saved-model "my_model" --output "my_model.onnx"
Convert frozen graph into onnx format
Frozen graph and saved model both have .pb extensions. If we have two files called keras_metadata.pb and saved_model.pb then this model is in save model format, we can just use the solution in the section above.
Even if the model is indeed a frozen graph, we can still convert it with tf2onnx. The command is below.
python -m tf2onnx.convert --input frozen_graph.pb --inputs <comma-delimited input names go here> --outputs <comma-delimited output names go here> --output frozen_graph.onnx
Again, we can use the Netron app to find the input and output names.
What if our model has multiple inputs/outputs?
A more complex model can have multiple inputs or outputs. For example, my age/gender predictor I mentioned above has two outputs, one for age and one for gender. Moreover, the gender classification actually feeds into the age prediction branch.
In this case, all we need to do is create an Output class with two properties.
public class Output
{
[VectorType(1)]
[ColumnName("gender_output/Sigmoid:0")]
public float[] Gender { get; set; }
[VectorType(1)]
[ColumnName("age_output/BiasAdd:0")]
public float[] Age { get; set; }
}
We can do the same if the model has multiple inputs; just add multiple properties into the Input class.
Conclusion
Even though I still prefer to train models using Python, ML.NET certainly opens new possibilities in the C# .NET world. The fact that it supports the onnx format also means that we can seamlessly integrate most pre-trained models into existing C# projects without much trouble.
Found this useful example, but I have a savedmodel, convert to onnx and use in ML.net. I use jpg/png and model
from TeachableMachine web. I works ok in their app but does not gives same results when exported, using same test image to predict.
Can you post your problem as a question on Stackoverflow and put the link here?
duongnt.bk, have you tried to use onnx for regression problems?
I have been looking into importing trained neural network to solve certain regression problem and implement it into C#, however I run into problem where the dimension comes as (-1, 68) for the input (or at least if I load the model that’s what I get)
Would you know what is the work around that? or can you suggest some approach?
Appreciate that.
In theory, for ML.NET there shouldn’t be any difference between a classification model and a regression model. Can you create a MCVE and post it on Stackoverflow?
I was able to figure it out. Just followed your tutorial much more closely and it works really well. Thank you 😉
You’re welcome. Glad that you figured it out.
Hi . is there way to achieve hierarchical classification in ml.net?
Hi Tushar,
Personally I haven’t tried hierarchical classification yet, but I think it should be possible.
Hello:
I am rather new to ML.NET and Tensorflow Model, but I have enough experience with C#.
I downloaded some pre-trained Tensorflow models, they came with .H5 model files and each of them has one Json format file for each model.
Here is one example:
{
“class_name”: “Sequential”,
“config”: {
“name”: “sequential”,
“layers”: [{
“class_name”: “Dense”,
“config”: {
“name”: “dense_1”,
“trainable”: true,
“batch_input_shape”: [null, 512],
“dtype”: “float32”,
“units”: 512,
“activation”: “linear”,
“use_bias”: true,
“kernel_initializer”: {
“class_name”: “GlorotUniform”,
“config”: {
“seed”: null
}
},
“bias_initializer”: {
“class_name”: “Zeros”,
“config”: {}
},
“kernel_regularizer”: null,
“bias_regularizer”: null,
“activity_regularizer”: null,
“kernel_constraint”: null,
“bias_constraint”: null
}
}, {
“class_name”: “LeakyReLU”,
“config”: {
“name”: “leaky_re_lu_14”,
“trainable”: true,
“dtype”: “float32”,
“alpha”: 0.20000000298023224
}
}, {
“class_name”: “Dense”,
“config”: {
“name”: “dense_2”,
“trainable”: true,
“dtype”: “float32”,
“units”: 512,
“activation”: “linear”,
“use_bias”: true,
“kernel_initializer”: {
“class_name”: “GlorotUniform”,
“config”: {
“seed”: null
}
},
“bias_initializer”: {
“class_name”: “Zeros”,
“config”: {}
},
“kernel_regularizer”: null,
“bias_regularizer”: null,
“activity_regularizer”: null,
“kernel_constraint”: null,
“bias_constraint”: null
}
}, {
“class_name”: “LeakyReLU”,
“config”: {
“name”: “leaky_re_lu_15”,
“trainable”: true,
“dtype”: “float32”,
“alpha”: 0.20000000298023224
}
}, {
“class_name”: “Dense”,
“config”: {
“name”: “dense_3”,
“trainable”: true,
“dtype”: “float32”,
“units”: 512,
“activation”: “linear”,
“use_bias”: true,
“kernel_initializer”: {
“class_name”: “GlorotUniform”,
“config”: {
“seed”: null
}
},
“bias_initializer”: {
“class_name”: “Zeros”,
“config”: {}
},
“kernel_regularizer”: null,
“bias_regularizer”: null,
“activity_regularizer”: null,
“kernel_constraint”: null,
“bias_constraint”: null
}
}, {
“class_name”: “LeakyReLU”,
“config”: {
“name”: “leaky_re_lu_16”,
“trainable”: true,
“dtype”: “float32”,
“alpha”: 0.20000000298023224
}
}, {
“class_name”: “Dense”,
“config”: {
“name”: “dense_4”,
“trainable”: true,
“dtype”: “float32”,
“units”: 512,
“activation”: “linear”,
“use_bias”: true,
“kernel_initializer”: {
“class_name”: “GlorotUniform”,
“config”: {
“seed”: null
}
},
“bias_initializer”: {
“class_name”: “Zeros”,
“config”: {}
},
“kernel_regularizer”: null,
“bias_regularizer”: null,
“activity_regularizer”: null,
“kernel_constraint”: null,
“bias_constraint”: null
}
}, {
“class_name”: “LeakyReLU”,
“config”: {
“name”: “leaky_re_lu_17”,
“trainable”: true,
“dtype”: “float32”,
“alpha”: 0.20000000298023224
}
}]
},
“keras_version”: “2.2.4-tf”,
“backend”: “tensorflow”
}
I want to know how I can get C# class definition for input and output?
I am using Visual Studio 2022 on Windows 10 (Version 21H2).
Thanks,
Hi John,
From your JSON file, I guess the input and output have been omitted.
Try to see if you can use the Netron app to load your model and view its architecture.
I am trying to replicate your code in Visual Studio but I am getting some errors:
The first one is in LoadFromEnumerable
Severity Code Description Project File Line Suppression State
Error CS0411 The type arguments for method ‘DataOperationsCatalog.LoadFromEnumerable(IEnumerable, SchemaDefinition)’ cannot be inferred from the usage. Try specifying the type arguments explicitly. ONNX.Test C:\Users\ben\source\repos\ONNX.Test\ONNX.Test\Program.cs 27 Active
The second error is in
Severity Code Description Project File Line Suppression State
Error CS0246 The type or namespace name ‘TResult’ could not be found (are you missing a using directive or an assembly reference?) ONNX.Test C:\Users\ben\source\repos\ONNX.Test\ONNX.Test\Program.cs 29 Active
I have tried specifying data types but could not make it compile properly. Can you give me a hand?
Thanks
Ben
here is my code:
using Microsoft.ML.Data;
using Microsoft.ML;
using NumSharp;
namespace ONNX.Test
{
internal class Program
{
static void Main(string[] args)
{
var modelPath = "fashionmnist_cnn_model.onnx";var outputColumnNames = new[] { "dense_1" };
var inputColumnNames = new[] { "conv2d_input" };
var mlContext = new MLContext();
var pipeline = mlContext.Transforms.ApplyOnnxModel(outputColumnNames, inputColumnNames, modelPath);
var content = np.Load_Npz<byte[,]>("test_image.npz");
var data = np.array(content["test_image.npy"])
.astype(NPTypeCode.Single)
.flatten() // remember to flatten the tensor
.ToArray<float>();
var input = new Input { Data = data };
var dataView = mlContext.Data.LoadFromEnumerable(input);
var transformedValues = pipeline.Fit(dataView).Transform(dataView);
var output = mlContext.Data.CreateEnumerable<TResult>(transformedValues, reuseRowObject: false);
}
public class Input
{
[VectorType(28 * 28 * 1)] // Our model takes a three-dimensional tensor as input but ML.NET takes a flatten vector as input
[ColumnName("conv2d_input")] // This name must match the input node's name
public float[] Data { get; set; }
}
public class Ouput
{
[VectorType(10)] // Because output node has (10) shape
[ColumnName("dense_1")] // This name must match the output node's name
public float[] Data { get; set; }
}
}
}
Hi Ben, sorry for the late response.
There is a mistake in my sample code. Please modify this line.
To this.
Notice that there is a typo in the output class name, which I have fixed. It should be
Outputinstead ofOuput.I got the following error on this line of code
var mlContext = new MLContext();
An unhandled exception of type ‘System.Collections.Generic.KeyNotFoundException’ occurred in mscorlib.dll
Additional information: The given key was not present in the dictionary. occurred
Can you help me what is the problem?
Hi Angela,
The line you posted only creates a
MLContextinstance and should not throw that error. Could you please post the full stack trace?using Microsoft.ML;
using Microsoft.ML.Data;
using NumSharp;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace ConsoleApp2
{
class Program{
static void Main(string[] args)
{
var modelPath = @"C:\Users\Deslab\source\repos\ConsoleApp2\ConsoleApp2\fashionmnist_cnn_model.onnx";
var outputColumnNames = new[] { "dense_1" };
var inputColumnNames = new[] { "conv2d_input" };
var mlContext = new MLContext();
var pipeline = mlContext.Transforms.ApplyOnnxModel(outputColumnNames, inputColumnNames, modelPath);
var content = np.Load_Npz<byte[,]>(@"C:\Users\Deslab\source\repos\ConsoleApp2\ConsoleApp2\test_image.npz");
var data = np.array(content["test_image.npz"])
.astype(NPTypeCode.Single)
.flatten() // remember to flatten the tensor
.ToArray<float>();
var input = new Input { Data = data };
var dataView = mlContext.Data.LoadFromEnumerable(inputColumnNames);
var transformedValues = pipeline.Fit(dataView).Transform(dataView);
var output = mlContext.Data.CreateEnumerable<Output>(transformedValues, reuseRowObject: false);
output.Single();
}
class Input
{
[Microsoft.ML.Data.VectorType(28 * 28 * 1)] // Our model takes a three-dimensional tensor as input but ML.NET takes a flatten vector as input
[ColumnName("conv2d_input")] // This name must match the input node's name
public float[] Data { get; set; }
}
class Output
{
[VectorType(10)] // Because output node has (10) shape
[ColumnName("dense_1")] // This name must match the output node's name
public float[] Data { get; set; }
}
}
}
I am using visual studio 2017 and .NET framework 4.6.1
Hi Angela, from your code I don’t see anything that can throw an exception at that line. Can you try switching to .NET Core 3.1 or .NET 6?
If that doesn’t work then please send the stack trace of the exception.