https://github.com/djhmateer/api-security-test test code for this article.

I’ve got a website which on the front page needs to make a request to another server which will do processing on a Python application.

The Python app takes around 8GB of RAM to run well (hate speech analyser with a large trained model).

So I want the Python app running on a different server.

To communicate I want to use HTTP as the remote server will be on my own hypervisor, and the web app is on Azure. So totally different networks. TCP is a perfect communication protocol for this.

The request will be a line of text, and the response will be a small amount of json.

Minimal API with .NET

Turning to the tools I know best, let’s see how easy it is to start

Out of the box the templates for .NET 6, minimal API’s, without OpenAPI (Swagger) support gives:

var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

app.UseHttpsRedirection();

var summaries = new[]
{
    "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
};

app.MapGet("/weatherforecast", () =>
{
    var forecast = Enumerable.Range(1, 5).Select(index =>
       new WeatherForecast
       (
           DateTime.Now.AddDays(index),
           Random.Shared.Next(-20, 55),
           summaries[Random.Shared.Next(summaries.Length)]
       ))
        .ToArray();
    return forecast;
});

app.Run();

internal record WeatherForecast(DateTime Date, int TemperatureC, string? Summary)
{
    public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);
}

And the output is some json.

Rick Anderson Tutorial on MS pares it down to:

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.MapGet("/", () => "Hello World!");

app.Run();

Which is an HTTP endpoint which returns hello world.

Need to remove the 2 launchUrl: Swagger lines from launch.json

# from windows cmd
curl -v https://localhost:7153/

Post and Get and Routing for ToDo

https://docs.microsoft.com/en-us/aspnet/core/tutorials/first-web-api?view=aspnetcore-6.0&tabs=visual-studio example of post and get for todo application.

Postman for testing

alt text

File, Settings, General, disable SSL Verification

  • HTTP method to POST
  • URI to https://localhost:7153/todoitems
  • Body tab, raw, type to JSON
{
  "name":"walk dog",
  "isComplete":true
}

click send and I get back

{
    "id": 1,
    "name": "walk dog",
    "isComplete": true
}

Have deployed the API to http://hmsoftware.org/ as a test server.

I’ve implemented a simple deployment pipeline so when I push to github, it pulls and rebuilds on the test server.

Have not done ssl certs yet.

It works POSTing to the test server.

Example

using System.Globalization;
using CsvHelper;
using CsvHelper.Configuration.Attributes;
using Serilog;

var builder = WebApplication.CreateBuilder(args);

builder.Logging.ClearProviders();

var logger = new LoggerConfiguration()
    .MinimumLevel.Information()
    .WriteTo.Console()
    .WriteTo.File("logs/information.log")
    .CreateLogger();

builder.Logging.AddSerilog(logger);

var app = builder.Build();

logger.Information("****Starting API");

// returns text
app.MapGet("/textget", () =>
    "hello from textget2"
);

app.MapGet("/jsonget", () =>
   // this will serialise the object and returns json by default as is of type T
   // https://docs.microsoft.com/en-us/aspnet/core/fundamentals/minimal-apis?view=aspnetcore-6.0#responses
   new { Message = "hello world from jsonget" }
);

app.MapPost("/hs", Handler3);
async Task<IResult> Handler3(HSDto hsdtoIn)
{
    // csv helper to write inbound hsdto to a csv
    var recordsToWrite = new List<HSDto>();
    recordsToWrite.Add(hsdtoIn);

    var guid = Guid.NewGuid();

    var path = "/home/dave/hatespeech";
    await using (var writer = new StreamWriter($"{path}/input/{guid}.csv"))
    await using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
    {
        csv.WriteRecords(recordsToWrite);
    }

    // poll the output directory
    var outputFile = $"{path}/output/{guid}.csv";
    while (true)
    {
        if (File.Exists(outputFile))
        {
            var hsdto = new HSDto();

            // found output file, convert to json object
            using (var reader = new StreamReader(outputFile))
            using (var csv = new CsvReader(reader, CultureInfo.InvariantCulture))
            {
                var records = csv.GetRecords<PythonDTO>();
                foreach (var record in records)
                {
                    logger.Information($"Text: {record.Text} ");
                    logger.Information($"Prediction: {record.Prediction} ");
                    logger.Information($"Score: {record.HateScore} ");

                    hsdto.Text = record.Text;
                    hsdto.Score = record.HateScore;
                    hsdto.Prediction = record.Prediction;
                }
            }

            // clean up
            File.Delete(outputFile);
            return Results.Json(hsdto);
        }

        await Task.Delay(100);
    }
}

app.Run();

class PythonDTO
{
    public string Text { get; set; }
    public string Prediction { get; set; }
    [Name("Hate score")]
    public string HateScore { get; set; }
}

class HSDto
{
    public string Text { get; set; }
    public string? Score { get; set; }
    public string? Prediction { get; set; }
}

Nopasswd


sudo visudo
dave ALL=(ALL) NOPASSWD: ALL

Run command

asdf

Foo

https://docs.microsoft.com/en-us/aspnet/core/tutorials/min-web-api?view=aspnetcore-6.0&tabs=visual-studio minimal API example.

No model binding

Good for microservice which is really what I’m doing.

REST?

https://docs.microsoft.com/en-us/aspnet/core/fundamentals/minimal-apis?view=aspnetcore-6.0#optional-parameters

Securing the API

I only want 1 specific server which runs https://osr4rightstools.org/ to call this API. It will call the API to do hatespeech processing from the Analyse Text Above button on the homepage.

  • OAuth2.0 - this is more suitable to users ie SSO (Single Sign On) trusted 3rd party like Google, Azure, AWS. Token exchange.

  • OpenID Connect. Built on top of OAuth2

  • Certificate based

  • API Key

Authorize

https://docs.microsoft.com/en-us/aspnet/core/fundamentals/minimal-apis?view=aspnetcore-6.0#authorization minimal web api can do it

app.MapGet("/auth", () => "This endpoint requires authorization")
   .RequireAuthorization();

Certificate Authentication

https://docs.microsoft.com/en-us/aspnet/core/security/authentication/certauth?view=aspnetcore-6.0

Microsoft.AspNetCore.Authentication.Certificate

https://github.com/dotnet/AspNetCore.Docs/issues/22054 possible nginx proxy code here.

alt text

My development cert?

Got a ERR_BAD_SSL_CLIENT_AUTH_CERT on chrome.

This is feeling very low level, and will needs to pass through the cert from my nginx reverse proxy.

Good practise

validate the input thoroughly before sending

use TLS

Rate limit

Use IP address filtering?

Use cloudflare?