Api
https://github.com/djhmateer/api-security-test test code for this article.
I’ve got a website which on the front page needs to make a request to another server which will do processing on a Python application.
The Python app takes around 8GB of RAM to run well (hate speech analyser with a large trained model).
So I want the Python app running on a different server.
To communicate I want to use HTTP as the remote server will be on my own hypervisor, and the web app is on Azure. So totally different networks. TCP is a perfect communication protocol for this.
The request will be a line of text, and the response will be a small amount of json.
Minimal API with .NET
Turning to the tools I know best, let’s see how easy it is to start
Out of the box the templates for .NET 6, minimal API’s, without OpenAPI (Swagger) support gives:
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.UseHttpsRedirection();
var summaries = new[]
{
"Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
};
app.MapGet("/weatherforecast", () =>
{
var forecast = Enumerable.Range(1, 5).Select(index =>
new WeatherForecast
(
DateTime.Now.AddDays(index),
Random.Shared.Next(-20, 55),
summaries[Random.Shared.Next(summaries.Length)]
))
.ToArray();
return forecast;
});
app.Run();
internal record WeatherForecast(DateTime Date, int TemperatureC, string? Summary)
{
public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);
}
And the output is some json.
Rick Anderson Tutorial on MS pares it down to:
var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();
app.MapGet("/", () => "Hello World!");
app.Run();
Which is an HTTP endpoint which returns hello world.
Need to remove the 2 launchUrl: Swagger lines from launch.json
# from windows cmd
curl -v https://localhost:7153/
Post and Get and Routing for ToDo
https://docs.microsoft.com/en-us/aspnet/core/tutorials/first-web-api?view=aspnetcore-6.0&tabs=visual-studio example of post and get for todo application.
Postman for testing
File, Settings, General, disable SSL Verification
- HTTP method to POST
- URI to https://localhost:7153/todoitems
- Body tab, raw, type to JSON
{
"name":"walk dog",
"isComplete":true
}
click send and I get back
{
"id": 1,
"name": "walk dog",
"isComplete": true
}
Have deployed the API to http://hmsoftware.org/ as a test server.
I’ve implemented a simple deployment pipeline so when I push to github, it pulls and rebuilds on the test server.
Have not done ssl certs yet.
It works POSTing to the test server.
Example
using System.Globalization;
using CsvHelper;
using CsvHelper.Configuration.Attributes;
using Serilog;
var builder = WebApplication.CreateBuilder(args);
builder.Logging.ClearProviders();
var logger = new LoggerConfiguration()
.MinimumLevel.Information()
.WriteTo.Console()
.WriteTo.File("logs/information.log")
.CreateLogger();
builder.Logging.AddSerilog(logger);
var app = builder.Build();
logger.Information("****Starting API");
// returns text
app.MapGet("/textget", () =>
"hello from textget2"
);
app.MapGet("/jsonget", () =>
// this will serialise the object and returns json by default as is of type T
// https://docs.microsoft.com/en-us/aspnet/core/fundamentals/minimal-apis?view=aspnetcore-6.0#responses
new { Message = "hello world from jsonget" }
);
app.MapPost("/hs", Handler3);
async Task<IResult> Handler3(HSDto hsdtoIn)
{
// csv helper to write inbound hsdto to a csv
var recordsToWrite = new List<HSDto>();
recordsToWrite.Add(hsdtoIn);
var guid = Guid.NewGuid();
var path = "/home/dave/hatespeech";
await using (var writer = new StreamWriter($"{path}/input/{guid}.csv"))
await using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
csv.WriteRecords(recordsToWrite);
}
// poll the output directory
var outputFile = $"{path}/output/{guid}.csv";
while (true)
{
if (File.Exists(outputFile))
{
var hsdto = new HSDto();
// found output file, convert to json object
using (var reader = new StreamReader(outputFile))
using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
{
var records = csv.GetRecords<PythonDTO>();
foreach (var record in records)
{
logger.Information($"Text: {record.Text} ");
logger.Information($"Prediction: {record.Prediction} ");
logger.Information($"Score: {record.HateScore} ");
hsdto.Text = record.Text;
hsdto.Score = record.HateScore;
hsdto.Prediction = record.Prediction;
}
}
// clean up
File.Delete(outputFile);
return Results.Json(hsdto);
}
await Task.Delay(100);
}
}
app.Run();
class PythonDTO
{
public string Text { get; set; }
public string Prediction { get; set; }
[Name("Hate score")]
public string HateScore { get; set; }
}
class HSDto
{
public string Text { get; set; }
public string? Score { get; set; }
public string? Prediction { get; set; }
}
Nopasswd
sudo visudo
dave ALL=(ALL) NOPASSWD: ALL
Run command
asdf
Foo
https://docs.microsoft.com/en-us/aspnet/core/tutorials/min-web-api?view=aspnetcore-6.0&tabs=visual-studio minimal API example.
No model binding
Good for microservice which is really what I’m doing.
REST?
Securing the API
I only want 1 specific server which runs https://osr4rightstools.org/ to call this API. It will call the API to do hatespeech processing from the Analyse Text Above
button on the homepage.
-
OAuth2.0 - this is more suitable to users ie SSO (Single Sign On) trusted 3rd party like Google, Azure, AWS. Token exchange.
-
OpenID Connect. Built on top of OAuth2
-
Certificate based
-
API Key
Authorize
https://docs.microsoft.com/en-us/aspnet/core/fundamentals/minimal-apis?view=aspnetcore-6.0#authorization minimal web api can do it
app.MapGet("/auth", () => "This endpoint requires authorization")
.RequireAuthorization();
Certificate Authentication
https://docs.microsoft.com/en-us/aspnet/core/security/authentication/certauth?view=aspnetcore-6.0
Microsoft.AspNetCore.Authentication.Certificate
https://github.com/dotnet/AspNetCore.Docs/issues/22054 possible nginx proxy code here.
My development cert?
Got a ERR_BAD_SSL_CLIENT_AUTH_CERT on chrome.
This is feeling very low level, and will needs to pass through the cert from my nginx reverse proxy.
Good practise
validate the input thoroughly before sending
use TLS
Rate limit
Use IP address filtering?
Use cloudflare?