General .NET – Brian Pedersen's Sitecore and .NET Blog

I just deployed, and now I cannot log in. Is it an error in the code? Or is it an error in data. Don’t worry, your login ticket just got messed up in the Properties table of the CORE database.

Sitecore Properties table in the CORE database — Tickets in the Properties table in the CORE database

To restore your login, simply delete your tickets from the Properties table in the CORE database:

delete  FROM [yoursitecore_Core].[dbo].[Properties]
  where Key like 'CORE_SC_TICKET_%'
  and Value like '%{login}%'

Replace the {login} with the login name, and you will be able to instantly login after deletion.

The complete error is:

9960 11:09:24 ERROR Application error.
Nested Exception
System.Web.HttpUnhandledException (0x80004005): Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Exception: String or binary data would be truncated.
The statement has been terminated. ---> System.Data.DataException: Error executing SQL command:  INSERT INTO [Properties] (   [Key], [Value] ) VALUES (   @name, @value ) ---> System.Data.SqlClient.SqlException: String or binary data would be truncated.
The statement has been terminated.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
   at Sitecore.Data.DataProviders.Sql.DataProviderCommand.ExecuteNonQuery()
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at Sitecore.Data.DataProviders.Sql.DataProviderCommand.ExecuteNonQuery()
   at Sitecore.Data.DataProviders.Sql.SqlDataApi.<>c__DisplayClass33_0.<Execute>b__0()
   at Sitecore.Data.DataProviders.NullRetryer.Execute[T](Func`1 action, Action recover)
   at Sitecore.Data.Properties.SqlPropertyStore.SetStringValueCore(String prefixedName, String value)
   at Sitecore.Data.Properties.PropertyStore.SetStringValue(String name, String value)
   at Sitecore.Web.Authentication.DefaultTicketManager.CreateTicket(String userName, String startUrl, Boolean persist)
   at Sitecore.Pipelines.LoggedIn.Ticket.Process(LoggedInArgs args)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Sitecore.Reflection.ReflectionUtil.InvokeMethod(MethodInfo method, Object[] parameters, Object obj)
   at Sitecore.Nexus.Pipelines.NexusPipelineApi.Resume(PipelineArgs args, Pipeline pipeline)
   at Sitecore.Pipelines.Pipeline.Start(PipelineArgs args, Boolean atomic)
   at Sitecore.sitecore.login.Default.LoggedIn()
   at Sitecore.sitecore.login.Default.LoginClicked(Object sender, EventArgs e)
   at System.Web.UI.WebControls.Button.OnClick(EventArgs e)
   at System.Web.UI.WebControls.Button.RaisePostBackEvent(String eventArgument)
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
   at System.Web.UI.Page.HandleError(Exception e)
   at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
   at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
   at System.Web.UI.Page.ProcessRequest()
   at System.Web.UI.Page.ProcessRequest(HttpContext context)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.<>c__DisplayClass285_0.<ExecuteStepImpl>b__0()
   at System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step)
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Taht’s it. You are now a Sitecore security expert.

MORE TO READ:

Sitecore 9 login: An Error Occurred, String or binary data would be truncated from Claus Fyhn
Error: An error occurred after logging in Sitecore from Vinicius Deschamps
Sitecore.Web.Authentication.c__DisplayClass5.b__2(Ticket ticket) +52 from briancaos

If you wish to poke around in the IdentityServer users database directly, the API seems a little bit fishy. This is because the direct data access are very low level, and consists of HttpClient extension methods. They are, in fact, so low level that you think that you are doing something wrong. But you are not, this is the way. So here’s the quick guide to getting started.

STEP 1: THE NUGET PACKAGES

Start with these packages:

IdentityModel – The extension methods for IdentityServer
Microsoft.Extensions.Http – The HttpClient and IHttpClientFactory

STEP 2: CREATE AN ACCESSTOKEN REPOSITORY

IdentityServer have 2 types of access: Client access is the one we use and will grant us access to all users, and user access with will grant individual users access to their own data.

You need to know the URL to your IdentityServer, the client ID (usually a neat readable string) and a client secret (usually an unreadable guid-like string):

using IdentityModel.Client;

namespace IdentityServer.Repositories
{
  public class AccessTokenRepository 
  {
    private readonly IHttpClientFactory _httpClientFactory;

    public AccessTokenRepository(IHttpClientFactory httpClientFactory)
    {
      _httpClientFactory = httpClientFactory;
    }

    public async Task<string> GetClientAccessTokenAsync()
    {
      var tokenRequest = new ClientCredentialsTokenRequest
      {
        Address = $"[IdentityServerUrl]/connect/token",
        ClientId = "[IdentityServer Client ID]",
        ClientSecret = "[IdentityServer Client Secret]"
      };

      var client = _httpClientFactory.CreateClient("HttpClient");
      var response = await client.RequestClientCredentialsTokenAsync(tokenRequest);
      if (response.IsError)
        throw new Exception($"{GetType()}.GetClientAccessToken failed: {response.ErrorDescription} ({response.HttpStatusCode}) ");

      return response.AccessToken;
    }
  }
}

STEP 3: CREATE A USER REPOSITORY

With an Client Access Token in hand, we are allowed to access the IdentityServer database.

using System.Net;
using System.Net.Http.Headers;

namespace IdentityServer.Repositories
{
  public class UserRepository 
  {
    private IHttpClientFactory _httpClientFactory;
    private AccessTokenRepository _accessTokenRepository;

    public UserRepository(IHttpClientFactory httpClientFactory, AccessTokenRepository accessTokenRepository) 
    {
      _httpClientFactory = httpClientFactory;
      _accessTokenRepository = accessTokenRepository;
    }

    public async Task<string?> GetUserAsync(string username)
    {
      var client = _httpClientFactory.CreateClient("HttpClient");
      client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", await _accessTokenRepository.GetClientAccessTokenAsync());

      var responseMessage = await client.GetAsync($"[IdentityServerUrl]/localapi/users/{username}");
      if (responseMessage.StatusCode == HttpStatusCode.NotFound)
        return null;  
      if (responseMessage.StatusCode == HttpStatusCode.Unauthorized)
        throw new Exception("Unauthorized");
      if (!responseMessage.IsSuccessStatusCode)
        throw new Exception($"{responseMessage.StatusCode}");
      var userJson = await responseMessage.Content.ReadAsStringAsync();
      return userJson;
    }
  }
}

The above code will return the bare-bone Json output from IdentityServer. You will need to parse the output accordingly. The reason this code does not do this is that you can design your own user database, so there is not really a standard data format.

STEP 4: USE THE USERREPOSITORY

The code expects an IHttpClientFactory, so we need to hook up such a thing, and we need to hook up the 2 repositories in our dependency injection framework. Hereafter, the repos will be available:

builder.Services.AddHttpClient("HttpClient");
builder.Services.AddSingleton<AccessTokenRepository>();
builder.Services.AddSingleton<UserRepository>();

...
...

// Example on how to use the userRepository:
if (await _userRepository.GetUserAsync(userName) == null)
  return BadRequest();

MORE TO READ:

So why would you ever log to Application Insights AND File log at the same time? Well, if you are hosting your own applications on your own machine, it can be great to have a file version of what’s happened. The file will be updated immediately, whereas Application Insights are sometimes up to 5 minutes behind.
Also, it gives you the opportunity to log at different levels. Application Insights is not cheap, so having Application Insights log only warnings and errors, but the file logging debug can be a money saver.
And, when developing, the file log is far superior to Application Insights.

My colleague Kim Schiøtt whiffed up this extension method to make the configuration easier.

STEP 1: THE NUGET PACKAGES

You need the following packages:

STEP 2: THE CONFIGURATION

This will configure the logging:

"ApplicationInsights": {
    "InstrumentationKey": "[The instrumentation key]"
  },
"Logging": {
    "PathFormat": "[The path and file format used in file logging, e.g.: c:\\log-{Date}.txt]",
    "LogLevel": {
      "Default": "Information"
    },
    "ApplicationInsightsLoggerProvider": {
      "LogLevel": {
        "Default": "Warning"
      }
    }
  }

The “Logging” section configures the different log levels for file and Application Insights.

STEP 3: THE EXTENSION METHOD

This method allows you to add logging easily:

using Microsoft.ApplicationInsights.AspNetCore.Extensions;

namespace MyCode
{
    public static class WebApplicationBuilderExtensions
    {
        public static void AddLogging(this WebApplicationBuilder builder)
        {
            // Add file logging
            builder.Host.ConfigureLogging(logging =>
                {
                    logging.AddFile(builder.Configuration.GetSection("Logging"));
                }
            );
            // Add Application Insights Logging
            var options = new ApplicationInsightsServiceOptions();
            options.InstrumentationKey = builder.Configuration["ApplicationInsights:InstrumentationKey"];
            builder.Services.AddApplicationInsightsTelemetry(options);
        }
    }
}

STEP 4: CALL THE METHOD WHEN BUILDING YOUR APPLICATION

This is an example where I configure my logging:

using System;
using Microsoft.AspNetCore.Builder;
using Microsoft.Extensions.Hosting;

var webApplicationOptions = new WebApplicationOptions();
var builder = WebApplication.CreateBuilder(webApplicationOptions);
builder.AddLogging();
...
...
...
builder.Build();

MORE TO READ:

Logging in .NET Core and ASP.NET Core from Microsoft
Filtering Application Insights telemetry using a ITelemetryProcessor by briancaos
.NET Core Worker Services with Application Insights and Serilog by briancaos
Application Insights logging with .NET from Microsoft
Double traces on Insight without WriteTo: File and WriteTo: Console from GitHub

Previously I wrote the article Write to file from multiple threads async with C# and .NET Core, which explains how to use a ConcurrentQueue to append lines to a text file. And the ConcurrentQueue is great for the fire-and-forget scenarios of log file writing, where you really do not read from the files you write.

But if you are reading the files you are writing, you need some kind of locking mechanism to ensure that the code block reading and writing the file are accessed by one thread at a time.

There are many options including lock, semaphors and ReaderWriterLock. In this article I will use the good old Mutex class.

STEP 1: THE CODE

using System;
using System.Threading;
using System.IO;
					
public class ThreadSafeFileWriter
{
	public string ReadFile(string filePathAndName)
	{
      // This block will be protected area
      using (var mutex = new Mutex(false, filePathAndName.Replace("\\", "")))
      {
        var hasHandle = false;
        try
        {
          // Wait for the muted to be available
          hasHandle = mutex.WaitOne(Timeout.Infinite, false);
          // Do the file read
          if (!File.Exists(filePathAndName))
            return string.Empty;
		  return File.ReadAllText(filePathAndName);
        }
        catch (Exception)
        {
          throw;
        }
        finally
        {
          // Very important! Release the mutex
          // Or the code will be locked forever
          if (hasHandle)
            mutex.ReleaseMutex();
        }
      }
	}
	
    public void WriteFile(string fileContents, string filePathAndName)
    {
      using (var mutex = new Mutex(false, filePathAndName.Replace("\\", "")))
      {
        var hasHandle = false;
        try
        {
          hasHandle = mutex.WaitOne(Timeout.Infinite, false);
          if (File.Exists(filePathAndName))
            return;
		  File.WriteAllText(filePathAndName, fileContents);
        }
        catch (Exception)
        {
          throw;
        }
        finally
        {
          if (hasHandle)
            mutex.ReleaseMutex();
        }
      }
    }	
}

SO WHAT’S UP WITH THE MUTEX?

The Mutex will let threads wait for the mutex handle in a queue until the mutex is released. So if 3 threads will write to the same file, they will wait in line nicely until they are granted access.

It is very important that the code releases the mutex as well, or the code will be locked forever.

Mutexes can be named, like in the example above. This locks the shared resource system wide. So if another process tries to access the same code, that process will also wait in line. Backslashes in mutex names are a reserved character and must be removed.

Taht’s it. You are now a multi-thread expert. Happy coding.

MORE TO READ:

Write to file from multiple threads async with C# and .NET Core by briancaos
Semaphores don’t have owners by Raymond Chen
Mutex class from Microsoft
TextWriter.Synchronized(TextWriter) Method from Microsoft
Thread-safe acces to a text file from CodeProject

There are several ways of batching lists and enumerables with C#. Here are 3 ways:

METHOD 1: USING MoreLINQ

MoreLINQ is an extension package for LINQ that implements yet another set of useful LINQ expressions like Shuffle, Pad, and Batch.

Get the MoreLINQ NuGet package here: https://www.nuget.org/packages/morelinq/

METHOD 2: BATHCING USING FOREACH AND YIELD

This implementation looks a lot like the one that MoreLINQ uses. It utilizes a foreach loop, building a new list that is then returned:

using System;
using System.Collections.Generic;
using System.Linq;

namespace MyCode
{
  internal static class EnumerableExtension
  {
    public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> enumerator, int size)
    {
      T[] batch = null;
      var count = 0;

      foreach (var item in enumerator)
      {
        if (batch == null)
          batch = new T[size];

        batch[count++] = item;
        if (count != size)
          continue;

        yield return batch;

        batch = null;
        count = 0;
      }

      if (batch != null && count > 0)
        yield return batch.Take(count).ToArray();
    }
  }
}

METHOD 3: BATCHING USING SKIP() AND TAKE()

This method is a little shorter, and utilizes the Skip() to jump to the next batch, and Take() to grab the batch:

using System;
using System.Collections.Generic;
using System.Linq;

namespace MyCode
{
  internal static class EnumerableExtension
  {
    public static IEnumerable<IEnumerable<T>> Batch<T>(this IEnumerable<T> enumerator, int size)
    {
      var length = enumerator.Count();
      var pos = 0;
      do
      {
        yield return enumerator.Skip(pos).Take(size);
        pos = pos + size; 
      } while (pos < length);
    }
  }
}

HOW TO USE THE BATCH METHODS ABOVE:

You can try it out using this tiny sample:

public class Program
{
	public static void Main()
	{
		// Add 25 numbers to a list 
        List<int> list = new List<int>();
		for (int i=0;i<25;i++)
			list.Add(i);
        // Batch the list into batches of 7
        var batches = list.Batch(7);
		foreach (var batch in batches)
		{
			foreach (var item in batch)
				Console.Write(item + " ");
			Console.WriteLine(" batch end");	
		}			
	}
}

The code above will return the following output:

0 1 2 3 4 5 6  batch end
7 8 9 10 11 12 13  batch end
14 15 16 17 18 19 20  batch end
21 22 23 24  batch end

That’s it. You are now a C# expert. Happy coding.

MORE TO READ:

MoreLINQ Github page
Create batches in linq from StackOverflow
Break up a list into batches with C# .NET from Andras Nemes
C# Lists in Lists – Getting all inner values, or the unique inner values by briancaos
C# Remove Duplicates from List with LINQ by briancaos
Generic lists and Predicates by briancaos

Namespaces in XML are hard to understand, I’ll admit that. But even I have learned that namespaces in XML documents matter. These elements are not the same:

<item>
    <title>Brian Caos</title>
    <description>A blog on C# coding</description>
    <link>https://briancaos.wordpress.com/</link>
</item>

<item xmlns:g="http://base.google.com/ns/1.0" version="2.0">
    <g:title>Brian Caos</g:title>
    <g:description>A blog on C# coding</g:description>
    <g:link>https://briancaos.wordpress.com/</g:link>
</item>

These 2 set of elements need to be mapped differently when deserialized in C#:

// This will map the <title> element in XML
[XmlElement("title"]
public string Title { get; set; }

// ... and this will map the <g:title> element in XML
[XmlElement("title", Namespace = "http://base.google.com/ns/1.0")]
public string Title { get; set; }

Unfortunately it is extremely common that businesses completely ignore the namespacing rules and send elements with or without namespaces. And we developers have to deal with it.

SO HOW DO WE FIX IT?

There is a class though, the XmlTextReader, that allows to be overwritten to ignore namespaces. Microsoft discourages us from using the class though. But it’s our only chance to ignore the namespace when deserializing.

STEP 1: CREATE A IgnoreNamespaceXmlTextReader CLASS

using System.Xml;

namespace MyCode
{
  public class IgnoreNamespaceXmlTextReader : XmlTextReader
  {
    public IgnoreNamespaceXmlTextReader(TextReader reader) : base(reader) 
    { 
    }

    public override string NamespaceURI => "";
  }
}

STEP 2: DO NOT SPECIFY ANY NAMESPACES IN THE C# MODEL CLASS

This is an example of how to deserialize a simple XML into a simple model class. The model class does not define any namespaces in the class attributes:

using System.Xml.Serialization;

namespace MyCode
{
  [XmlRoot(ElementName = "model")]
  [XmlType("model")]
  public class XmlModel
  {
    [XmlElement("name")]
    public string Name { get; set; }  

    [XmlElement("value")]
    public string Value { get; set; } 
  }
}

STEP 3: USE THE IgnoreNamespaceXmlTextReader WHEN DESERIALIZING XML

Now we can deserialize any XML we receive, namespace or no namespace:

// We can now import 
// an xml without namespaces
string s = @"<model>
               <name>hello</name>
               <value>world</value>
             </model>";

var sr = new StringReader(s);
var xmlSerializer = new XmlSerializer(typeof(XmlModel));
var xmlModel = (XmlModel)xmlSerializer.Deserialize(new IgnoreNamespaceXmlTextReader(sr));
// xmlModel.Name = "hello" and xmlModel.Value = "world"

// ---------------------------------------------------

// And we can also import 
// an xml with namespaces
// because the IgnoreNamespaceXmlTextReader ignores 
// any namespaces in the receiving XML
string s = @"<model xmlns:g=""http://base.google.com/ns/1.0"">
               <g:name>hello</g:name>
               <g:value>world</g:value>
             </model>";

var sr = new StringReader(s);
var xmlSerializer = new XmlSerializer(typeof(XmlModel));
var xmlModel = (XmlModel)xmlSerializer.Deserialize(new IgnoreNamespaceXmlTextReader(sr));
// xmlModel.Name = "hello" and xmlModel.Value = "world"

MORE TO READ:

XmlTextReader Class from Microsoft
C# Working with Namespaces in XDocument and XML by briancaos
Deserialize XML array to string[] and how to allow JSON to be deserialized into the same POCO class by briancaos

WebP is a new compressed image format developed by Google. If you want to convert WebP to any other format, you seem to be mostly out of luck if you are using C#.

There is, however, some tools out there. Google have released 2 DLL’s, libwebp_x64.dll and libwebp_x86.dll, but these DLL’s are running native unmanaged code, so to utilize these, we need to run our code on a Windows machine, and we need to wrap the code so it can be used from managed code.

Jose Pinero have done exactly this, wrapped the DLL’s so you can call the functionalities from your C# managed code.

So here is the journey on how to convert a WebP image to a JPEG using C#.

STEP 1: DOWNLOAD libwebp_x64.dll AND libwebp_x86.dll AND WebPWrapper.cs

Go to the following GIT repo and get the WebPWrapper.cs. Don’t forget the 2 Google DLL’s as well, libwebp_x64.dll and libwebp_x86.dll:

https://github.com/JosePineiro/WebP-wrapper

STEP 2: INCLUDE SYSTEM.DRAWING

System.Drawing is the old image library native to Windows. Include the package:

https://www.nuget.org/packages/System.Drawing.Common/

STEP 3: MAKE AN JPEG EXTENSION FOR THE IMAGE CLASS

The C# Image class can easily be extended to create a JPEG image:

using System.Drawing;
using System.Drawing.Imaging;
using Encoder = System.Drawing.Imaging.Encoder;

namespace WebPConverter
{
  public static class ImageExtensions
  {
    public static void SaveJpeg(this Image img, string filePath, long quality)
    {
      var ep = new EncoderParameters(1);
      ep.Param[0] = new EncoderParameter(Encoder.Quality, quality);
      img.Save(filePath, GetEncoder(ImageFormat.Jpeg), ep);
    }

    public static void SaveJpeg(this Image img, Stream stream, long quality)
    {
      var ep = new EncoderParameters(1);
      ep.Param[0] = new EncoderParameter(Encoder.Quality, quality);
      img.Save(stream, GetEncoder(ImageFormat.Jpeg), ep);
    }

    private static ImageCodecInfo GetEncoder(ImageFormat format)
    {
      ImageCodecInfo[] codecs = ImageCodecInfo.GetImageDecoders();
      return codecs.Single(codec => codec.FormatID == format.Guid);
    }
  }
}

STEP 4: CONVERT THE WEBP IMAGE TO JPEG

This simple code will convert a WebP image to a Jpeg image:

using System.Drawing;
using WebPConverter;
using WebPWrapper;

WebP webp = new WebP();
Bitmap bitmap = webp.Load("d:\\1.webp");
bitmap.SaveJpeg("d:\\1.jpeg", 80);

RUNNING THE WebPWrapper.cs AS A SERVICE

Jose Pinero have tied in some GUI related code in the WebPWrapper.cs code, so if you need to run the code as a service, an API, or as a worker process you need to clean up some of his code.

The easiest way to do this is to remove the following line from the WebPWrapper.cs file and handle whatever compiler error is appearing, by deleting the lines that cast an error.

using System.Windows.Forms;

CONVERTING AND UPLOADING TO BLOB STORAGE:

This example is larger, as I am grabbing a WebP image from a website, converting it into a JPEG, and storing that image in a Blob storage:

using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using System.Drawing;
using WebPConverter;
using WebPWrapper;

// Grab a WebP image from a Google Website
HttpClient client = new HttpClient();
var response = await client.GetAsync("https://www.gstatic.com/webp/gallery/1.webp");

// Open a connection to a Blob Container
CloudBlobContainer cloudBlobContainer = CloudStorageAccount.Parse(YOUR_CONNECTiONSTRING).CreateCloudBlobClient().GetContainerReference("containerName");

// Give the blob a name and create a blob reference
string blobName = "MediaStorage/test/image1.jpg";
var blobReference = cloudBlobContainer.GetBlockBlobReference(blobName);
blobReference.Properties.ContentType = "image/jpg";

// Convert the WebP image to a JPEG and stream it into the Blob file we created
await using var jpgStream = await ConvertWebPContentToJpegContent(response.Content);
jpgStream.Seek(0, SeekOrigin.Begin);
await blobReference.UploadFromStreamAsync(jpgStream);

async Task<Stream> ConvertWebPContentToJpegContent(HttpContent content)
{
  // Grab the raw WebP image
  var rawWebP = await content.ReadAsByteArrayAsync();
  // Convert the WebP image to a JPEG
  Bitmap bitmap = new WebP().Decode(rawWebP);
  MemoryStream memoryStream = new MemoryStream();
  // Save the JPEG to a Stream that can be saved in a blob storage
  bitmap.SaveJpeg(memoryStream, 80);
  return memoryStream;
}

FINAL NOTES:

I’m not sure why it’s so hard to convert WebP to JPEG using C#, if it’s because WebP is not widely used, or WebP is so superior that no one would ever convert to another format. The fact that the code is tied to a Windows machine can be a deal breaker for some. There are paid products out there that maybe will help you.

That’s it. You are now a WebP expert. Happy coding.

MORE TO READ:

Google documentation on WebP
Google download repository
Jose Pinero Webp-Wrapper code on Github
WebP Images in C# and ASP.NET from Programming in Csharp
What is WebP? Pros and cons of this next-gen image format by Matt Ellis
LibWebP – converting from PNG to WebP

Azure Managed Identity is Microsoft’s solution to manage credentials for you, eliminating the need to store usernames, passwords, certificates and other secrets in your config files.

Basically, with Managed Identity you establish a trust between a server and a resource. For example, if you have a SQL server and a Web Server, you use managed identity to grant the Web Server access to the SQL server, so you don’t need to keep a username/password to the SQL server in the connectionstring.

Read here on how to configure managed identity in Azure.

Once the managed identity have been established between the SQL server and the Web Server, you will have a connection string and a managed identity client id. Notice that the connectionstring does not contain a username and a password:

  "ConnectionStrings": {
    "SqlConnection": "Server=tcp:[servername].database.windows.net,1433;Initial Catalog=[databasename]",
    "ClientId": "[guid]"
  },

Now, remember that the managed identity is a trust between the web server and the sql server. If you develop code locally, you still need a username/password in your connectionstring, as it is not possible to establish a trust between a local machine and a Azure Sql Server.

The easiest way to overcome this is to have a local .config file without a client id, and in the code check if the client id is empty. If it’s empty, you connection the old fashioned way, if not, you use the client id to connect.

ENOUGH TALK, SHOW ME THE CODE

This is an example of a connection manager that uses the client id if it exists to connect using Azure Managed Identity:

using System.Data;
using System.Data.SqlClient;
using Azure.Identity;
using Microsoft.Extensions.Configuration;

namespace MyCode
{
    public class ConnectionManager : IConnectionManager
    {
        private readonly IConfiguration _configuration;
        private readonly string _connectionString;

        public ConnectionManager(IConfiguration configuration)
        {
            _configuration = configuration;
            _connectionString = _configuration.GetConnectionString("SqlConnection");
        }

        public IDbConnection CreateConnection()
        {
            SqlConnection? connection;
            string clientId = _configuration.GetConnectionString("ClientId");
			
            // Empty client id means we are running towards a local 
		    // database using username/password. Connect the old
            // fashioned way
            if (string.IsNullOrWhiteSpace(clientId))
              return new SqlConnection(_connectionString);
            
            // Client id is set. Use the Managed Identity Client Id to 
			// establish a connection
			connection = new SqlConnection(_connectionString);
            var credential = new DefaultAzureCredential(new DefaultAzureCredentialOptions { ManagedIdentityClientId = clientId });
            var token = credential.GetToken(new Azure.Core.TokenRequestContext(new[] { "https://database.windows.net/.default" }));
            connection.AccessToken = token.Token;
            return connection;
        }
    }
}

The connectionmanager is injected into your service registration:

services.AddTransient<IConnectionManager, ConnectionManager>();

And the connectionmanager are now ready to be used in your SQL repositories:

namespace MyCode
{
    public class MyRepository 
    {
        private readonly IConnectionManager _connectionManager;

        public BannerRepository(IConnectionManager connectionManager)
        {
            _connectionManager = connectionManager;
        }

        public async Task MockupSQLAsync()
        {
            using var connection = _connectionManager.CreateConnection();
            await connection.ExecuteAsync("some sql", ...);
        }
    }
}

That’s it. You are now an expert in Managed Identity. Happy coding.

MORE TO READ:

Connecting from your application to resources without handling credentials from Microsoft
What are managed identities for Azure resources? from Microsoft
C# Using Dapper as your SQL framework in .NET Core by briancaos
Azure Identity client library for .NET – version 1.6.1 from Microsoft

A classic issue in Sitecore: Some code running in one Site Context is creating a link to a page in another context, and the link turns out to be complete garbage.

For example, this code is running in the modules_shell context and is trying to create a link to a page on my main website (the website context):

var site = Sitecore.Sites.SiteContext.Current.Name;
var item = Sitecore.Context.ContentDatabase.GetItem("/sitecore/content/Website/Frontpage/live");
var link = Sitecore.Links.LinkManager.GetItemUrl(item);

// site = modules_shell
// link = /sitecore-modules/shell/Website/Frontpage/Live

That link is definitely not correct, so you try adding some ItemUrlBuilderOptions to the LinkManager.GetItemUrl():

var site = Sitecore.Sites.SiteContext.Current.Name;
var item = Sitecore.Context.ContentDatabase.GetItem("/sitecore/content/Website/Frontpage/live");
Sitecore.Links.ItemUrlBuilderOptions options = new Sitecore.Links.ItemUrlBuilderOptions();
options.Site = Sitecore.Sites.SiteContext.GetSite("website");
options.LanguageEmbedding = Sitecore.Links.LanguageEmbedding.Never;
options.SiteResolving = true;
var link = Sitecore.Links.LinkManager.GetItemUrl(item, options));

// site = modules_shell
// link = /Live

This is much better. The link is now relative to the website base url. But we need the link to be absolute to the base domain.

To achieve this, make sure you have the targetHostName and scheme set in the <sites> setting in your config:

<site name="website" 
 rootPath="/sitecore/content/website" startItem="/frontpage" 
 targetHostName="mywebsite.com" 
 scheme="https"
 ...
 ...
/>

Then use this little nifty method:

private static string Qualify(string relativeUrl, string sitename)
{
  SiteContext sitecontext = SiteContext.GetSite(sitename);
  return Qualify(relativeUrl, sitecontext);
}

private static string Qualify(string relativeUrl, SiteContext sitecontext)
{
  if (!relativeUrl.StartsWith("/"))
    relativeUrl = "/" + relativeUrl;
  return string.Format("{0}://{1}{2}", sitecontext.SiteInfo.Scheme, sitecontext.TargetHostName, relativeUrl);
}

And your output will be fully qualified:

var item = Sitecore.Context.ContentDatabase.GetItem("/sitecore/content/Website/Frontpage/live");
Sitecore.Links.ItemUrlBuilderOptions options = new Sitecore.Links.ItemUrlBuilderOptions();
options.Site = Sitecore.Sites.SiteContext.GetSite("website");
options.LanguageEmbedding = Sitecore.Links.LanguageEmbedding.Never;
options.SiteResolving = true;
var link = Sitecore.Links.LinkManager.GetItemUrl(item, options);
var fullyQualifiedLink = Qualify(link, "website");

// link = /Live
// fullyQualifiedLink = https://mywebsite.com/Live

That’s it. You are now a Sitecore expert. happy coding.

MORE TO READ:

Build item and media URLs from Sitecore
Sitecore Links with LinkManager and MediaManager by briancaos
Sitecore LinkField TargetItem is NULL – what’s wrong? by briancaos

DevOps is a complicated discipline, so when you as a developer have to set up a build pipeline in Azure Devops you are sometimes in deep water.

Thats why I made this yml template that I can copy from whenever I need to build a .net 6 project:

trigger:
  - main

pool:
  name: Azure Pipelines
  vmImage: 'windows-2019'

variables:
  buildConfiguration: 'Release'

steps:
- task: UseDotNet@2
  displayName: 'Use .NET Core 6'
  inputs:
    version: 6.0.x

- task: DotNetCoreCLI@2
  displayName: Restore
  inputs:
    command: restore
    projects: '**/*.csproj'

- task: DotNetCoreCLI@2
  displayName: Build
  inputs:
    projects: '**/*.csproj'

- task: DotNetCoreCLI@2
  displayName: Publish
  inputs:
    command: publish
    publishWebProjects: false
    projects: '**/*.csproj'
    arguments: '--configuration release --output $(build.artifactstagingdirectory) /property:PublishWithAspNetCoreTargetManifest=false'
    zipAfterPublish: false

- task: PublishBuildArtifacts@1
  displayName: 'Publish Artifact'
  inputs:
    ArtifactName: myproject

I name the file “azure-pipelines.yml” and place it in the root of my project:

azure-pipelines.yml added to the root of my project

When the pipeline has run it creates an artifact that can be deployed:

SO WHAT DOES EACH SECTION DO?

trigger	Determines which branch will trigger the pipeline. Is used if you set up continuous deployment
pool	Determines the server that it uses a build server. Use windows-2019, not ubuntu if you wish to build .net core 6 code
variables	Sets up build variables
task: UseDotNet@2	Tells the build to use .net core 6 when building. If you leave out this step, it will build using .net core 3.1 instead.
DotNetCoreCLI@2/Restore	Gets NuGet packages
DotNetCoreCLI@2/Build	Builds the project
DotNetCoreCLI@2/Publish	Copies the files to a folder. publishWebProjects will determine what to copy. If true, it will copy web projects zipAfterPublish determines if the out should be zipped. property:PublishWithAspNetCoreTargetManifest=false will copy your NuGet dll’s along with the dlls of the project
PublishBuildArtifacts@1	Creates an artifact that can be used by your release pipeline

This is just an introduction to DevOps which is a separate discipline to development. I hope that the file will give you a hint of what to do before panicking if you are given the task of setting up your own build and release pipelines.

Happy DevOps’ing.

MORE TO READ:

Force ‘dotnet publish’ to publish dependencies using PublishWithAspNetCoreTargetManifest by Benjamin Day
Build, test, and deploy .NET Core apps from Microsoft
.NET Core CLI task by Microsoft
Publish Build Artifacts task by Microsoft

I added exception handling to my .net core 6 API, and any of my error messages just throws an error 500.

The complete exception is as follows:

System.NotSupportedException: Serialization and deserialization of 'System.IntPtr' instances are not supported. Path: $.TargetSite.MethodHandle.Value.
 ---> System.NotSupportedException: Serialization and deserialization of 'System.IntPtr' instances are not supported.
   at System.Text.Json.Serialization.Converters.UnsupportedTypeConverter`1.Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options)
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.GetMemberAndWriteJson(Object obj, WriteStack& state, Utf8JsonWriter writer)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryWrite(Utf8JsonWriter writer, T value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.GetMemberAndWriteJson(Object obj, WriteStack& state, Utf8JsonWriter writer)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryWrite(Utf8JsonWriter writer, T value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.GetMemberAndWriteJson(Object obj, WriteStack& state, Utf8JsonWriter writer)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryWrite(Utf8JsonWriter writer, T value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
   at System.Text.Json.Serialization.JsonConverter`1.WriteCore(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)

This is my controller. As you can see, it’s supposed to throw an error 400 (BadRequest):

namespace MyController
{
  [ApiController]
  [Route("api/public/v{version:apiVersion}/[controller]")]
  public class MyController : Controller
  {
    [HttpGet("{id}")]
    public async Task<IActionResult> GetData([FromRoute] int id)
    {
      try
      {
        // .. do stuff
        // .. do stuff
        // .. do stuff
      }
      catch (Exception ex)
      {
        return BadRequest(ex);
      }
    }
  }
}

But all it does is throw an error 500:

Any exception is thrown as an error 500, with the same error message.

THE PROBLEM:

It turns out, that the built in System.Text.Json serializer cannot serialize an exception. So when I return the BadRequest(ex), ex cannot be serialized to a JSON text string. And since my API is JSON based, I will get an error.

THE SOLUTION:

Do not return an exception. Instead, return an object than can be serialized, like the ProblemDetails class. I made an extension method for this purpose:

using Microsoft.AspNetCore.Mvc;
using System.ComponentModel.DataAnnotations;
using System.Net;
using System.Security.Authentication;

namespace MyCode
{
  public static class ExceptionExtensions
  {
    public static ProblemDetails ToProblemDetails(this Exception e)
    {
      return new ProblemDetails()
      {
        Status = (int)GetErrorCode(e.InnerException ?? e),
        Title = e.Message
      };
    }

    private static HttpStatusCode GetErrorCode(Exception e)
    {
      switch (e)
      {
        case ValidationException _:
          return HttpStatusCode.BadRequest;
        case FormatException _:
          return HttpStatusCode.BadRequest;
        case AuthenticationException _:
          return HttpStatusCode.Forbidden;
        case NotImplementedException _:
          return HttpStatusCode.NotImplemented;
        default:
          return HttpStatusCode.InternalServerError;
      }
    }

  }
}

With this extension method I can easily convert the Exception to a ProblemDetails class:

namespace MyController
{
  [ApiController]
  [Route("api/public/v{version:apiVersion}/[controller]")]
  public class MyController : Controller
  {
    [HttpGet("{id}")]
    public async Task<IActionResult> GetData([FromRoute] int id)
    {
      try
      {
        // .. do stuff
        // .. do stuff
        // .. do stuff
      }
      catch (Exception ex)
      {
        return BadRequest(ex.ToProblemDetails());
      }
    }
  }
}

And whenever an error occurs, it will return the proper type and message:

Now returning a proper error 400 with a nice title

That’s it. You are now a .net core 6 API exception handling expert. Happy coding.

MORE TO READ:

Why does System.Text.Json throw a `NotSupportedException` for `System.IntPtr` when I’m not using `System.IntPtr`? from StackOverflow
Resolved: Why does System.Text.Json throw a `NotSupportedException` for `System.IntPtr` when I’m not using `System.IntPtr`? by Isaac
ProblemDetails Class from Microsoft
.NET Core Api – Catch exceptions using a middleware by briancaos
.NET Core Catch Model Binding Exceptions by briancaos
ASP.Net Core API – “‘s’ is an invalid start of a value. Path: $ | LineNumber: 0 | BytePositionInLine: 0.” by briancaos

Azure Table Storage is a NoSQL key-value store that can store massive amounts of data in a semi-structured way. Data is stored using a PartitionKey/RowKey keyset and a list of values. So far so good, but how do we query the danm thing?

First things first, have a look at my table storage, a list of famous celebrities:

STEP 1: THE NUGET PACKAGES

You need the following NuGet package to access a Table Storage:

Azure.Data.Tables

STEP 2: MAP THE TABLE STORAGE TO A MODEL CLASS:

Querying Table Storages are done by creating a model that implements the ITableEntity interface. The interface ensures that the ETag and Timestamp is present in the class:

using Azure;
using Azure.Data.Tables;

namespace MyCode
{
  public class FamousCelebritiesModel : ITableEntity
  {
    public string PartitionKey { get; set; }
    public string RowKey { get; set; }
    public int CelebrityIndex { get; set; }
    public string RealName { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string Url { get; set; }
    public ETag ETag { get; set; } = default!;
    public DateTimeOffset? Timestamp { get; set; } = default!;
  }
}

STEP 3: CONNECT TO THE TABLE STORAGE

Connection is easy:

using Azure;
using Azure.Data.Tables;

namespace MyCode
{
  public class FamousCelebritiesRepository
  {
    private TableServiceClient _tableServiceClient;
    private TableClient _tableClient;

    public FamousCelebritiesRepository()
    {
      _tableServiceClient = new TableServiceClient("[your connection string]");
      _tableClient = _tableServiceClient.GetTableClient(tableName: "FamousCelebrities");
    }
	
	// more functions goes here
	// more functions goes here
	// more functions goes here
	// more functions goes here
	// more functions goes here

  }
}

QUERY 1: QUERYASYNC TO GET ALL ITEMS FROM THE TABLE STORAGE

The QueryAsync looks different from what we are used to, as the “await” operator is on the foreach loop:

public async Task<IEnumerable<FamousCelebritiesModel>> GetAllAsync()
{
  IList<FamousCelebritiesModel> modelList = new List<FamousCelebritiesModel>();
  var celebs = _tableClient.QueryAsync<FamousCelebritiesModel>(filter: "", maxPerPage: 10);
  await foreach (var celeb in celebs)
  {
    modelList.Add(celeb);
  }
  return modelList;
}

QUERY 2: QUERYASYNC WITH FILTER

You can apply simple LINQ filters to a Table Storage. This example filters by PartitionKey:

public async Task<IEnumerable<FamousCelebritiesModel>> GetByKnownAsync(string partitionKey)
{   
  IList<FamousCelebritiesModel> modelList = new List<FamousCelebritiesModel>();
  var celebs = _tableClient.QueryAsync<FamousCelebritiesModel>(x => x.PartitionKey == partitionKey, maxPerPage: 10);
  await foreach (var celeb in celebs)
  {
    modelList.Add(celeb);
  }
  return modelList;
}

QUERY 3: PAGINATION

Table Storage with millions of rows must be queried using pagination. Table Storage uses a “continuationtoken” to tell which page is next. Start with an empty token (null) to get the first page, and then use the continuationtoken from that query to get the next page:

public async Task<Tuple<string, IEnumerable<IFamousCelebritiesModel>>?> GetAllAsync(string continuationToken)
{
  IList<FamousCelebritiesModel> modelList = new List<FamousCelebritiesModel>();
  var celebs = _tableClient.QueryAsync<FamousCelebritiesModel>(filter: "", maxPerPage: 10);

  await foreach (var page in celebs.AsPages(continuationToken))
  {
    return Tuple.Create<string, IEnumerable<IFamousCelebritiesModel>>(page.ContinuationToken, page.Values);
  }
  return null;
}

Don’t know how to use a Tuple?

// Get first page:
var page = await GetAllAsync(null);
string continuationToken = page.Item1;
foreach (var item in page.Item2)  
{
  // take items from the list, for example:
  // item.RealName
}

// Get next page:
page = await GetAllAsync(continuationToken);
continuationToken = page.Item1;
foreach (var item in page.Item2)  
{
  // take items from the list, for example:
  // item.RealName
}

That’s it. You are now a Azure Table Storage Expert. Happy coding.

MORE TO READ:

Azure Table Storage – QueryAsync from Andy Sprague
TableClient.QueryAsync Method from Microsoft
How to Filter the query result for pagination in TableClient.QueryAsync() [Azure.Data.Tables] from Stackoverflow
What is Azure Table storage? from Microsoft
Quickstart: Azure Cosmos DB for Table for .NET from Microsoft

So the goal here is to define that a specific API endpoint in my .NET Core API requires an authorization token. It’s not all my endpoints that requires authentication, only those I specify.

To obtain this, I would like to decorate my api method in my controller with an attribute stating that this endpoint requires an authorization token in the request header before you are allowed to call it. Like this:

[HttpGet()]
[AuthTokenRequired]
public async Task<IActionResult> GetProfile()
{
  // Here goes my code...
}

The “AuthTokenRequired” is an attribute defined by me by combining a TypeFilterAttribute with an IAuthorizationFilter. This combination allows me to create a custom TypeFilterAttribute attribute while still using dependency injection in the class implmenting the IAuthorizationFilter interface.

But enough talk, let’s code.

THE SAMPLE CODE:

You need to see the code in it’s entirely to understand what’s happening. There are 2 classes here:

using Microsoft.AspNetCore.Mvc.Filters;
using Microsoft.AspNetCore.Mvc;
using System.Security.Authentication;
using Users.AuthenticationToken;

namespace MyCode
{
  // This is the attribute class. This allows you to 
  // decorate endpoints in your controllers with a
  // [AuthTokenRequired] attribute
  public class AuthTokenRequiredAttribute : TypeFilterAttribute
  {
    public AuthTokenRequiredAttribute() : base(typeof(TokenAuthorizationFilter))
    {
    }
  }

  // This is the actual code that runs the authorization. By splitting
  // the code into 2 classes, you can inject your dependencies into this class  
  [AttributeUsage(AttributeTargets.Method, Inherited = false, AllowMultiple = true)]
  public class TokenAuthorizationFilter : Attribute, IAuthorizationFilter
  {
	// Sample private variables, yours will look different
	private readonly SomeSampleRepository _rep;
    private readonly IWebHostEnvironment _environment;

    public TokenAuthorizationFilter(SomeSampleRepository rep, IWebHostEnvironment environment)
    {
      // This is just some sample code. This is added 
	  // to show you that you can now use dependency injection
	  // in this class.
	  // In your code you will add the classed needed to 
	  // do the proper authorization for you.
      _rep = rep; 
      _environment = environment;
    }
  
    public void OnAuthorization(AuthorizationFilterContext context)
    {
      // Here you will add your authorization code. This is just an 
	  // example of the possibilities in this method. Do your
	  // method the way you need it to work.
	  
	  if (_environment.IsDevelopment())
        return;
	  
	  // You can for example retrieve a request header and check the value
      var token = context.HttpContext.Request.Headers["AuthToken"];
      if (string.IsNullOrEmpty(token))
        throw new AuthenticationException("Unauthorized");
      if (token != "my-token")
        throw new AuthenticationException("Unauthorized");
    }
  }
}

So first we define the attribute itself. The class is called AuthTokenRequiredAttribute which will create an [AuthTokenRequired] attribute to be used in the api endpoints in your controller.

The attribute instantiates a new class, TokenAuthorizationFilter, and it is this class that actually does the authorization.

When decorating an API endpoint with the [AuthTokenRequired] attribute, the OnAuthorization method will be called each time before the API endpoint is called.

You must implement your own authorization logic here, I just included some inspiration on how to do it.

My controller class looks like this:

using Microsoft.AspNetCore.Mvc;

namespace MyController
{
  [ApiController]
  [Route("api/[controller]")]
  public class ProfileController : Controller
  {
    [HttpGet()]
    [AuthTokenRequired]
    public async Task<IActionResult> GetProfile()
    {
      try
      {
		// If the [AuthTokenRequired] fails, this
		// code will never be called. A AuthenticationException
		// will be thown before this code is reached
		// Pseudocode: do some code to get the profile and return it
        HttpRequest request = this.Request;
        return Ok(profile);
      }
      catch (Exception ex)
      {
        return BadRequest();
      }
    }
  }
}

That’s it. You are now an authorization expert. Happy coding.

MORE TO READ:

TypeFilterAttribute Class from Microsoft
IAuthorizationFilter Interface from Microsoft
Dependency Injection in IAuthorizationFilter ASP.Net Core 3 from StackOverflow
How to use ASP.NET Core MVC built-in Filters from GeeksArray
.NET Core Api – Catch exceptions using a middleware from briancaos
.NET Core Catch Model Binding Exceptions from briancaos

The following error may occur when upgrading Microsoft ApplicationInsights:

[FileNotFoundException: Could not load file or assembly ‘System.Diagnostics.DiagnosticSource, Version=4.0.4.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51’ or one of its dependencies. The system cannot find the file specified.]
Microsoft.AspNet.TelemetryCorrelation.TelemetryCorrelationHttpModule.Application_BeginRequest(Object sender, EventArgs e) +0
System.Web.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +142
System.Web.<>c__DisplayClass285_0.b__0() +38
System.Web.HttpApplication.ExecuteStepImpl(IExecutionStep step) +11857717
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +93

Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.8.4494.0

The error may happen in legacy .NET 4.x applications.

This is a conflict between Microsoft.AspNet.TelemetryCorrelation and Microsoft.ApplicationInsights, The 2 packages have different versions of System.Diagnostics.DiagnosticSource.

Microsoft.AspNet.TelemetryCorrelation use v4.0.4.0
Microsoft.ApplicationInsights use v5.0.0.0

To fix this, add a dependentAssembly to your web.config in the assemblyBinding section:

<dependentAssembly>
  <assemblyIdentity name="System.Diagnostics.DiagnosticSource" publicKeyToken="cc7b13ffcd2ddd51"  culture="neutral"/>
  <bindingRedirect oldVersion="0.0.0.0-5.0.0.0" newVersion="5.0.0.0"/>
</dependentAssembly>

Hope this will help you. Happy coding.

MORE TO READ:

System.IO.FileLoadException: Could not load file or assembly ‘System.Diagnostics.DiagnosticSource Version=4.0.4.0 from Github

Microsoft have released a new System.CommandLine library to handle parameters and arguments in your .NET Core 6 console applications.

The NuGet Package is currently (March 2nd 2023) in beta, so you need to allow prerelease NuGet packages in your solution.

But here’s how to use it.

STEP 1: THE NUGET PACKAGE

Include the following package:

System.Commandline

STEP 2: THE PROGRAM.CS

My code differs slightly from the one Microsoft suggest, as I am dividing my code into application configuration and application business logic. The division allows me to treat my business logic as a class with constructor based dependency injection. This is the bare necessities:

using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using System.CommandLine;

// ----------------------------------------------------
// My configuration part of the application
// ----------------------------------------------------
var host = Host.CreateDefaultBuilder(args)
    .ConfigureAppConfiguration((hostingContext, config) =>
    {
      config.AddJsonFile("appsettings.json");
	  // Add more config files if needed
    }
    )
    .ConfigureServices((hostingContext, services) =>
    {
	  // The "Application" service is the application
	  // itself. Add all other dependencies hereafter
      services.AddSingleton<Application>();
	  
    })
    .ConfigureLogging((hostingContext, logging) =>
      // Add more logging if needed
	  logging.AddConsole()
    )
  .Build();

// Run the application
var application = host.Services.GetRequiredService<Application>();
await application.ExecuteAsync(args);

// ----------------------------------------------------
// This is the business logic of my application.
// It's here that I define what the application will do
// and which parameters it requires
// ----------------------------------------------------
public class Application
{
  private readonly ILogger<Application> _logger;

  public Application(ILogger<Application> logger)
  {
    _logger = logger;
  }

  // This is where the arguments are defined
  public async Task<int> ExecuteAsync(string[] args)
  {
    var argument1 = new Option<string>(
        name: "--argument1",
        description: "This is my first argument that I allow"
    );

    var argument2 = new Option<int>(
        name: "--argument2",
        description: "This is another argument. This time I only allow an integer"
    );

    var rootCommand = new RootCommand("Commands");
    rootCommand.AddOption(argument1);
    rootCommand.AddOption(argument2);

    rootCommand.SetHandler(async (argument1, argument2) =>
		{
		  await RunApplicationAsync(argument1, argument2);
		},
        argument1, argument2
	);

    return await rootCommand.InvokeAsync(args);
  }

  private async Task RunApplicationAsync(string argument1, int argument2)
  {
	// This is the actual starting point of my business logic
    _logger.LogError("Hello world! {argument1} {argument2}", argument1, argument2);
    await Task.Delay(5000);
  }
}

EXPLANATION:

The first section is the typical setup of dependency injection, logging, etc. In line 31,32 I execute the application business login, which is the “Application” class.

Having a separate “Application” class that handles the business logic allows me to threat my application as any other class and use constructor based dependency injection.

The “ExecuteAsync” in line 49 is the entry point of my business logic, but all it does is setting up the arguments that my application allows. Please note that the argument names argument1 and argument2 are repeated several times, but this is the way.

So the actual application business logic starts at the bottom, in the RunApplicationAsync method (line 75). Because the System.CommandLine was hooked up in the ExecuteAsync method, the arguments can now be passed as arguments to my function.

From that point on, it’s business as usual, and your code will look no different from any other application.

HOW TO CALL YOUR APPLICATION:

Your application now allows arguments:

# Get help. This will not execute the RunApplicationAsync method
./application.exe -?

# Run the application with arguments
./application.exe --argument1 "mystring" --argument2 42

That’s it. You are now a command line expert. Happy coding.

MORE TO READ:

System.CommandLine overview from Microsoft
Tutorial: Get started with System.CommandLine from Microsoft
C# Log to Application Insights and File from your .NET 6 Application by briancaos
Getting Started with System.CommandLine by Matthew Leibowitz
Demystifying System.CommandLine by Kevin Bost

I was calling a stored procedure using Dapper, and for reasons beyond my understanding, the stored procedure returned strings as non-trimmed strings:

Strings are not trimmed when returned from my stored procedure

I could of course look into the stored procedure, but that would require me to obtain new knowledge of SQL, so instead I quickly jumped on the C# bandwagon. Good for me, because Dapper have thought of this scenario, and implemented a TypeHandler that can be globally applied to all Dapper query results automatically.

STEP 1: CREATE A NEW TYPEHANDLER

using Dapper;
using System.Data;

namespace MyCode
{
  public class TrimStringHandler : SqlMapper.TypeHandler<string>
  {
    public override string Parse(object value)
    {
      return (value as string)?.Trim();
    }

    public override void SetValue(IDbDataParameter parameter, string value)
    {
      parameter.Value = value;
    }
  }
}

This TypeHandler handles values of type string, and will trim the result when returned.

STEP 2: APPLY THE TYPEHANDLER GLOBALLY

We’re used to applying stuff using dependency injection, but in Dapper’s case, it’s just a global setting. Add the following line of code to your Program.cs:

Dapper.SqlMapper.AddTypeHandler(new TrimStringHandler());

And now all strings will be returned in trimmed form:

The strings are now nicely trimmed — Strings are nicely trimmed now

That’s it. You are now a Dapper expert. Happy coding.

MORE TO READ:

C# Using Dapper as your SQL framework in .NET Core by briancaos
Create C# Class from SQL by briancaos
Go beyond Dapper limits and map arbitrary complex class by Davide Mauri
Dapper repo at GitHub

Imagine this classic text where you need to replace tokens with some values:

Welcome {{user}}. Click on this link to confirm your subscription: {{url}}.

Tokens encapsulated in double brackets {{token}} are commonly used to mark tokens to be replaced with real values.

It’s simple to make your own token replacer:

using System.Collections.Generic;

namespace MyCode
{
  public static class TokenReplacer
  {
    public static string ReplaceTokens(this string s, Dictionary<string, string> tokens)
    {
      if (tokens == null)
        return s;

      foreach (var token in tokens)
        s = s.Replace("{{" + token.Key + "}}", token.Value);

      return s;
    }
  }
}

Usage is easy:

var tokenValues = new System.Collections.Generic.Dictionary<string, string>();
tokenValues.Add("user", "briancaos");
tokenValues.Add("url", "https://briancaos.wordpress.com/");
string text = "Welcome {{user}}. Click on this link to confirm your subscription: {{url}}.";
Console.WriteLine(text.ReplaceTokens(tokenValues));

// Output is:
// Welcome briancaos. Click on this link to confirm your subscription: https://briancaos.wordpress.com/.

That’s it. Happy coding.

MORE TO READ:

Replace {x} tokens in strings from stackoverflow
C# Remove specific Querystring parameters from URL from briancaos

The C# HttpClient is the go-to library when you need to GET, PUT, POST or DELETE from an API Endpoint. But one of the issues with HttpClient is that it needs to be instantiated as a singleton. This is a big no-no:

public static async Task<string> Get(string queryString)
{
  using (var httpClient = new HttpClient())
  {
    using (var result = await httpClient.GetAsync($"https://briancaos.wordpress.com"))
	{
	  string content = await result.Content.ReadAsStringAsync();
	  return content;
	}
  }
}

The HttpClient is a shared object, and disposing the HttpClient for each call will lead to socket exhaustion.

SO WHAT TO DO THEN?

The trick is to use a IHttpClientFactory. This will create static instances for you (as well as allowing you to use Polly and other middlewares, but that’s a story for another time).

It’s very easy to use:

STEP 1: INSTANTIATE AN IHttpClientFactory AT STARTUP:

To instantiate an IHttpClientFactory, use builder.Services.AddHttpClient:

var builder = WebApplication.CreateBuilder(webApplicationOptions);
...
builder.Services.AddHttpClient("HttpClient");
...
var app = builder.Build();
await app.RunAsync();

The IHttpClientFactory is named, and you can have several IHttpClientFactory instances, each with their own name. If you don’t know why you should have more then one, then you don’t need more than one.

STEP 2: INJECT THE IHttpClientFactory INTO YOUR CONSTRUCTOR

This is the basics of how your repository should look:

namespace MyCode
{
  public class MyRepository
  {
    private IHttpClientFactory _httpClientFactory;

    public MyRepository(IHttpClientFactory httpClientFactory)
    {
      _httpClientFactory = httpClientFactory;
    }

    public async Task<string> GetAsync(string url)
    {
      var client = _httpClientFactory.CreateClient("HttpClient");

      var response = await client.GetAsync($"{url}");
      if (!response.IsSuccessStatusCode)
        throw new Exception($"{response.StatusCode}: {response.ReasonPhrase}");
      var response = await response.Content.ReadAsStringAsync();
	  return response;
    }
  }
}

The _httpClientFactory.CreateClient(“HttpClient”); will ensure that you reuse the httpclient already created at startup.

That’s it. You are now a HttpClient expert. Happy coding.

MORE TO READ:

Add a UserAgent to the IHttpClientFactory in .NET Core from briancaos
HttpClient retry on HTTP timeout with Polly and IHttpClientBuilder from briancaos
HttpClient retry mechanism with .NET Core, Polly and IHttpClientFactory from briancaos
Using C# HttpClient from Sync and Async code from briancaos
YOU’RE USING HTTPCLIENT WRONG AND IT IS DESTABILIZING YOUR SOFTWARE by Simon Timms
Improper Instantiation antipattern from Microsoft
Make HTTP requests using IHttpClientFactory in ASP.NET Core from Microsoft

So, most API endpoints return JSON anyway, right? So why not make a method that can make a GET call to an API and return the response as an object directly?

It’s actually not that hard:

STEP 1: MAKE A MODEL CLASS

My Catfact API endpoint returns the following JSON:

{
  "fact": "All cats need taurine in their diet to avoid blindness. Cats must also have fat in their diet as they are unable to produce it on their own.",
  "length": 140
}

So my model class is equally simple:

public class CatModel
{
  public string Fact  { get; set; }
  public int Length { get; set; } 
}

STEP 2: MAKE A HTTPCLIENTREPOSITORY

using Microsoft.AspNetCore.Http;
using Newtonsoft.Json;

namespace MyCode
{
  public class HttpClientRepository
  {
    private IHttpClientFactory _httpClientFactory;

    public HttpClientRepository(IHttpClientFactory httpClientFactory)
    {
      _httpClientFactory = httpClientFactory;
    }

    public async Task<T?> GetAsync<T>(string url)
    {
      var client = _httpClientFactory.CreateClient("HttpClient");
      var response = await client.GetAsync(url);
      if (!response.IsSuccessStatusCode)
        throw new Exception($"{response.StatusCode}: {response.ReasonPhrase}");
      var responseObject = await response.Content.ReadFromJsonAsync<T>();
      return responseObject;
    }

  }
}

The method “GetAsync” deserializes the return value from the API endpoint to the model of a Generic Type “T“.

A Generic Type is a placeholder for a specific type that you define later. Using a Generic Type ensures that you can call this method regardless of the model returned, as long as the JSON can be deserialized to that model.

In other words: If your specific type matches the JSON returned, everything is fine.

STEP 3: HOW TO USE IT

In your Program.cs, add an IHttpClient and the HttpClientRepository:

builder.Services.AddHttpClient("httpClient");
builder.Services.AddSingleton<HttpClientRepository>();

To call the method is quite simple. This function returns the cat fact string:

public async Task<string?> GetCatFactAsync()
{
  CatModel? catFact = await _httpRepository.GetAsync<CatModel>("https://catfact.ninja/fact");
  return catFact?.Fact;
}

Not familiar with async coding? Don’t worry, just call the method it in the not-recommended way:

public string? GetCatFact()
{
  CatModel? catFact = _httpRepository.GetAsync<CatModel>("https://catfact.ninja/fact").Result;
  return catFact?.Fact;
}

But do yourself a favor and read up on async coding.

That’s it. You are now a HttpClient expert. Happy coding.

MORE TO READ:

C# HttpClient and IHttpClientFactory in .net core from briancaos
HttpClient retry on HTTP timeout with Polly and IHttpClientBuilder from briancaos
C# HttpClient POST or PUT Json with content type application/json from briancaos
Json.net, the home of Newtonsoft.JSON

Caching the response of an API speeds up the API endpoint because the code that generates the response is not called, but rather fetched from memory. This is especially helpful for API’s where the response is time-consuming to retrieve but the response does not change.

To add a response cache, you need to declare that caching is enabled, then specify the cache parameters for each of your API endpoints.

STEP 1: ENABLE CACHING IN Program.cs

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddControllers();
builder.Services.AddResponseCaching();  
var app = builder.Build();
app.UseHttpsRedirection();
// UseCors must be called before UseResponseCaching
// app.UseCors();
app.UseResponseCaching();
app.UseAuthorization();
app.MapControllers();
app.Run();

It is important that you call UseResponseCaching AFTER you have called UseCors.

STEP 2: DEFINE THE CACHE PROPERTIES FOR YOUR API ENDPOINT

To declare the cache properties for an endpoint, use the [ResponseCache] attribute:

using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;

namespace MyCode.Controllers
{
  [Route("api/[controller]")]
  [ApiController]
  public class MyController : ControllerBase
  {
    public MyController()
    {
    } 

    [HttpGet]
    [ResponseCache(Duration=10)]
    public string? GetCatFact()
    {
      // Pseudocode. Call something that will return a random cat fact
      // Insert your own code logic here:
      CatModel? catFact = _httpRepository.GetAsync<CatModel>("https://catfact.ninja/fact").Result;
      return catFact?.Fact;
    }
  }
}

In the example above, the ResponseCache attribute will cache the response for 10 seconds unconditionally.

The first request will call the underlying code, but the next requests are fetched from the memory cache, until the cache expires after 10 seconds. The response header will now have a max-age of 10, indicating the cache duration:

Response headers now has the cache-control max-age of 10, indicating that this response is cached in 10 seconds. — API Response as seen in Swagger

VARY THE CACHE BY HEADERS OR QUERYSTRING

The your API endpoint response is controlled using querystrings, you should add those as variation:

[HttpGet]
[ResponseCache(Duration=30, VaryByQueryKeys=new[] {"id", "search"})]
public string? GetCatFact([FromQuery] int id, [FromQuery] string search)
{
  // Still pseudo code, add your own logic
  CatModel? catFact = _httpRepository.GetAsync<CatModel>("https://catfact.ninja/fact").Result;
  return catFact?.Fact + id + search;
}

The response will very depending of the value of the querystrings. Use VaryByHeaders to vary by headers.

DEFINING CACHE PROFILES

Instead of defining the cache individually per endpoint, you can create cache profiles that can be reused. Go to the Program.cs and add the profiles to the AddControllers method:

builder.Services.AddControllers(options => 
  {
    options.CacheProfiles.Add("Default", 
      new CacheProfile() { Duration = 10 }
    );
    options.CacheProfiles.Add("Client",
      new CacheProfile() { Location = ResponseCacheLocation.Client, Duration = 10 }
    );
  }
);

Then use the profile name in the API endpoint. You can still decorate the endpoint with VaryByHeader/VaryByQueryKeys:

  [HttpGet]
  [ResponseCache(CacheProfileName="Default")]
  public string? GetCatFact()
  {
    CatModel? catFact = _httpRepository.GetAsync<CatModel>("https://catfact.ninja/fact").Result;
    return catFact?.Fact;
  }

  [HttpGet("2")]
  [ResponseCache(CacheProfileName="Client", VaryByQueryKeys=new[] {"id", "search"})]
  public string? GetCatFact2([FromQuery] int id, [FromQuery] string search)
  {
    CatModel? catFact = _httpRepository.GetAsync<CatModel>("https://catfact.ninja/fact").Result;
    return catFact?.Fact + id + search;
  }

That’s it. You are now a caching expert. Happy coding.

MORE TO READ:

Response caching in ASP.NET Core from Microsoft
Response Caching Middleware in ASP.NET Core from Microsoft
.NET API CORS: Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header.NET API from briancaos
C# .net Swagger API Versioning – Show versions in your Swagger page from briancaos

Sitecore System.Data.DataException: Error executing SQL command: INSERT INTO [Properties] ( [Key], [Value] ) VALUES ( @name, @value ) —> System.Data.SqlClient.SqlException: String or binary data would be truncated

Get Users from IdentityServer using .NET Core

C# Log to Application Insights and File from your .NET 6 Application

C# Thread Safe File Writer and Reader

C# List Batch – Braking an IEnumerable into batches with .NET

C# Ignoring Namespaces in XML when Deserializing

C# Convert WebP to JPEG in .NET

C# SQL Connection Using Azure Managed Identity

Sitecore Create Fully Qualified Url

Build .net core 6 projects with Azure Devops using yml files

Serialization and deserialization of ‘System.IntPtr’ instances are not supported. Path: $.TargetSite.MethodHandle.Value

C# Azure Table Storage QueryAsync, Paging and Filtering

C# Authorization in .NET Core API’s using Attributes, TypeFilterAttribute and IAuthorizationFilter

Could not load file or assembly ‘System.Diagnostics.DiagnosticSource, Version=4.0.4.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51’ or one of its dependencies. The system cannot find the file specified

Parameters and Arguments in .NET Core Command Line Applications

C# Dapper Trim String Values

C# String Token Replacer

C# HttpClient and IHttpClientFactory in .net core

C# Use HttpClient to GET JSON from API endpoint

.NET Core Caching in your API using AddResponseCaching and ResponseCache attribute