Note: phiên bản Tiếng Việt của bài này ở link dưới.

https://duongnt.com/polly-retry-request-vie

Occasionally, our HTTP requests might fail because of some transient errors. In most cases, such errors will go away if we retry later. While it is possible to write that retry logic ourselves, I prefer to use the Polly library instead.

You can download the sample code in this article from the link below.

https://github.com/duongntbk/PollyDemo

Manually retry a request

Assume we call an API that occasionally returns a 503-Service Unavailable error. We want to retry our requests whenever we get that code, but we also want to send the same request no more than five times before giving up. The code to achieve those goals seems easy enough at first glance.

private readonly HttpClient _client = new HttpClient(); 

public async Task<HttpResponseMessage> SendRequestWithRetry()
{
    HttpResponseMessage response;
    var sendCount = 0;
    do
    {
        var request = new HttpRequestMessage(HttpMethod.Get, new Uri("https://www.example.com"));
        response = await _client.SendAsync(request);
        sendCount++;
    }
    while (sendCount < 5 && response.StatusCode == System.Net.HttpStatusCode.ServiceUnavailable);

    return response;
}

However, it’s hard to reuse the code above for different endpoints. We need to extract the logic for sending HTTP requests to a new class, and add the retry logic to the SendRequest method of that class.

Also, it’s not a good idea to resend a failed request right away. Maybe we should introduce a delay to give the API some time to recover, etc. Perhaps you got the idea. It is time-consuming and error-prone to write all that retry logic ourselves. It’s much better to have a dedicated library handling this, something like Polly.

Retry requests with the Polly library

Preparation

You can install Polly by running the following command.

dotnet add package Polly

We also need a dummy API. I created a simple one for demonstration purposes, you can find it here. Navigate into the DummyApi folder and start the Api with the following command.

dotnet run

You can then access this API at https://localhost:5001.

How Polly works

Broadly speaking, we make requests with Polly via two interfaces; ISyncPolicy for synchronous calls, and IAsyncPolicy for asynchronous calls. In this article, we only care about the IAsyncPolicy interface. In particular, we will use the following method.

Task<TResult> ExecuteAsync<TResult>(Func<Context, Task<TResult>> action, Context context);

With it, we can simplify our retry logic to this.

var policy = <code to create an object that implements IAsyncPolicy>;
var pollyContext = new Context('<a name>');
var response = await policy.ExecuteAsync(async ctx =>
{
    var request = new HttpRequestMessage(HttpMethod.Get, new Uri("https://www.example.com"));
    return await _client.SendAsync(request);
}

And we are done. By configuring our policy object, we can do all the following things, and then some.

• Handling a particular type of exception, or exceptions that have a particular property.
• Retry requests up to a number of times, with an adaptive delay between each retry.
• Time out requests after a set period.
• Cache the responses.
• Limit request rate.
• Any combination of the above.

Retry a request after getting a particular error code

The actual code

As a first example, we will use Polly to retry requests that resulted in a 503 code up to five times. And we will wait 500 ms between each delay. The code is below.

var pollyContext = new Context("Retry 503");
var policy = Policy
    .Handle<HttpRequestException>(ex => ex.Message.Contains("503"))
    .WaitAndRetryAsync(
        5,
        _ => TimeSpan.FromMilliseconds(500),
        (result, timespan, retryNo, context) =>
        {
            Console.WriteLine($"{context.OperationKey}: Retry number {retryNo} within " +
                $"{timespan.TotalMilliseconds}ms. Original status code: 503");
        }
    );

var response = await policy.ExecuteAsync(async ctx =>
{
    var request = new HttpRequestMessage(HttpMethod.Get, new Uri("https://www.example.com"));
    var response = await _client.SendAsync(request);
    response.EnsureSuccessStatusCode();
    return response;
}, pollyContext);

What does the code do?

We will go through each line and see what they do.

The Handle<T> method determines which events our policy can handle. In this case, we use HttpRequestException as the generic type, and pass a delegate to filter only HttpRequestException that has the status code 503-Service Unavailable.

.Handle<HttpRequestException>(ex => ex.Message.Contains("503"))

The WaitAndRetryAsync method indicates what action we want to perform when the Handle method has a hit. In this case, we want to retry the requests after a short delay. WaitAndRetryAsync also has many overloads, but we use a simple one with just 3 arguments.

• retryCount: the maximum number of times to retry a request. We passed 5 here.
• Func<int, TimeSpan> sleepDurationProvider: a delegate to provide the delay time for a retry, based on the number of retries that have been performed earlier. We fix the delay to 500 ms here. But nothing stops us from using adaptive delay.

  _ => TimeSpan.FromMilliseconds(500)
  

• Action<Exception, TimeSpan, int, Context> onRetry: an action to be performed before the retry. We simply log the number of retries here.

  (ex, timespan, retryNo, context) =>
  {
      Console.WriteLine($"{context.OperationKey}: Retry number {retryNo} within " +
          $"{timespan.TotalMilliseconds}ms. Original status code: 503");
  }
  

You might also notice that we call EnsureSuccessStatusCode inside ExecuteAsync‘s delegate. This is because a HttpClient actually doesn’t throw an exception when the server returns a 503-Service Unavailable. However, the type in the Handle<T> method must be an exception. Because of that, we need to throw a HttpRequestException ourselves.

response.EnsureSuccessStatusCode();

Let’s try it out

The /demo/error/{requestId}/{errorCode} endpoint on the dummy API always returns errorCode the first time it is called with a particular requestId. All subsequent requests with the same requestId will succeed.

var requestId = Guid.NewGuid();
var errorCode = 503;
var response = await policy.ExecuteAsync(async ctx =>
{
    var request = new HttpRequestMessage(HttpMethod.Get, new Uri($"https://localhost:5001/demo/error/{requestId}/{errorCode}"));
    var response = await _client.SendAsync(request);
    response.EnsureSuccessStatusCode();
    return response;
}, pollyContext);

Console.WriteLine(await response.Content.ReadAsStringAsync());

The code above should print this to the command line.

Retry503: Retry number 1 within 503ms. Original status code: 503 // the first request returned 503 and was retried
be21236d-6eb7-4a58-8208-56ac1dee4207 did not throw an error. // the second request succeeded

Retry all transient errors

The code in the previous section allows us greater flexibility to control which status code we want to retry. But if we want to retry all transient HTTP errors, we will need to provide and maintain a complex delegate to the Handle<T> method. Instead, we can use the HandleTransientHttpError method inside the Polly.Extensions.Http library. This is how to create a policy to handle all 5xx codes, as well as 408-Request Timeout code.

var policy = HttpPolicyExtensions
    .HandleTransientHttpError() // <- an extension method from Polly.Extensions.Http
    .WaitAndRetryAsync(
        // omitted
    );

The code to use the new policy can also be simplified. Notice that we no longer need to call the EnsureSuccessStatusCode method.

var response = await policy.ExecuteAsync(async ctx =>
{
    var request = new HttpRequestMessage(HttpMethod.Get, new Uri($"https://localhost:5001/demo/error/{requestId}/{errorCode}"));
    return await _client.SendAsync(request);
}, pollyContext);

Console.WriteLine(await response.Content.ReadAsStringAsync());

The code above should print this to the command line.

RetryError: Retry number 1 after 500ms. Original status code: ServiceUnavailable // the first request returned 503 and was retried
be21236d-6eb7-4a58-8208-56ac1dee4207 did not throw an error. // the second request succeeded

Timeout a long-running request

It’s not mandatory to call WaitAndRetryAsync when creating a policy. Let’s say we have a request that can occasionally take a very long time. We prefer to terminate it after a set amount of time. The TimeoutAsync method can help us to do just that.

var timeoutInMilliSecs = 200; // Time out the request after 200 ms
var policy = Policy.TimeoutAsync(TimeSpan.FromMilliseconds(timeoutInMilliSecs), TimeoutStrategy.Pessimistic);

If a request made with this policy takes longer than 200 ms, Polly will automatically throw a TimeoutRejectedException.

We will test this new policy by calling the following endpoint: /demo/error/{requestId}/{delayInMilliSecs}. It will take at least delayInMilliSecs the first time it is called with requestId. All subsequent requests with the same requestId will be returned immediately.

// Will throw a TimeoutRejectedException
var requestId = Guid.NewGuid();
var delay = 1000; // The API will wait at least 1000 ms before returning
var response = await policy.ExecuteAsync(async ctx =>
{
    var request = new HttpRequestMessage(HttpMethod.Get, new Uri($"https://localhost:5001/demo/delay/{requestId}/{delay}"));
    return await _client.SendAsync(request);
}, pollyContext);

Combining multiple Polly policies

In this section, we will try to time out a long-running request and then retry it. We already know how to do each of those things, but how can we combine them together? The Policy.WrapAsync method is the answer. This method can create an AsyncPolicyWrap object that contains all IAsyncPolicy we pass to it. And because it also implements IAsyncPolicy, we can just drop it into our current code.

var timeoutPolicy = Policy.TimeoutAsync(TimeSpan.FromMilliseconds(timeoutInMilliSecs), TimeoutStrategy.Pessimistic);
var retryPolicy = Policy
    .Handle<TimeoutRejectedException>() // notice that we are handling a TimeoutRejectedException
    .WaitAndRetryAsync(
        // omitted
    );
var wrapper = Policy.WrapAsync(retryPolicy, timeoutPolicy);

var response = await wrapper.ExecuteAsync(async ctx =>
{
    var request = new HttpRequestMessage(HttpMethod.Get, new Uri($"https://localhost:5001/demo/delay/{requestId}/{delay}"));
    return await _client.SendAsync(request);
}, pollyContext);

Console.WriteLine(await response.Content.ReadAsStringAsync());

An important thing to keep in mind is when calling WrapAsync, we pass retryPolicy before timeoutPolicy. This is because AsyncPolicyWrap will wrap policies on the left outside policies on the right. When we pass (retryPolicy, timeoutPolicy), long-running requests will throw TimeoutRejectedException because of the timeoutPolicy first. Then retryPolicy will retry the TimeoutRejectedException. If we reverse this order, retryPolicy will be executed first, and it won’t be able to find any TimeoutRejectedException to retry.

The code above will print this to the command line.

RetryDelay: Retry number 1 after 500ms because The delegate executed asynchronously through TimeoutPolicy did not complete within the timeout. // This request took longer than 200 ms and was retried
be21236d-6eb7-4a58-8208-56ac1dee4207 was not delayed. // the second request succeeded

Conclusion

The Polly library makes retrying failed requests much easier. But it’s also capable of a lot more than that. From the signature of ExecuteAsync, keen-eyed readers might already have noticed that HTTP requests are not the only thing Polly can handle. In the next article, we will find out how to create our own custom policy with Polly.