Writing logs into Elastic with NLog , ELK and .Net 5.0 (Part 2)

In the previous article I demonstrated how we can write our log in our dotnet applications into Elastic with NLog, if you haven't read that article, I highly recommend reading it first.

After I published the first part, I received some comments that said, when they try to write logs into Elastic they have faced some problems.

I checked some of the cases and realized, they have enabled Basic authentication in Elastic. So, whenever NLog tries to write logs into Elastic by calling the endpoint of the Elastic, they get an authentication error.

OriginalException: Elasticsearch.Net.ElasticsearchClientException: Failed to ping the specified node.. Call: Status code 401 from: HEAD /
---> Elasticsearch.Net.PipelineException: Failed to ping the specified node.
---> Elasticsearch.Net.PipelineException: Could not authenticate with the specified node. Try verifying your credentials or check your Shield configuration.
at Elasticsearch.Net.RequestPipeline.ThrowBadAuthPipelineExceptionWhenNeeded(IApiCallDetails details, IElasticsearchResponse response)
at Elasticsearch.Net.RequestPipeline.Ping(Node node)
--- End of inner exception stack trace ---
at Elasticsearch.Net.RequestPipeline.Ping(Node node)
at Elasticsearch.Net.Transport`1.Ping(IRequestPipeline pipeline, Node node)
at Elasticsearch.Net.Transport`1.Request[TResponse](HttpMethod method, String path, PostData data, IRequestParameters requestParameters)
--- End of inner exception stack trace ---
# Audit exception in step 1 PingFailure:
Elasticsearch.Net.PipelineException: Could not authenticate with the specified node. Try verifying your credentials or check your Shield configuration.
at Elasticsearch.Net.RequestPipeline.ThrowBadAuthPipelineExceptionWhenNeeded(IApiCallDetails details, IElasticsearchResponse response)
at Elasticsearch.Net.RequestPipeline.Ping(Node node).

To solve the problem you should set credentials in your nlog.config file. so change the nlog.config file like this.

requireAuth="true"
username="*******"
password="********"

After I added these attributes to my target everything was working well. But the other issue reported was the SSL connection problem. If your Elastic endpoint has SSL, you may also face this error.

Elasticsearch.Net.ElasticsearchClientException: Failed to ping the specified node.. Call: Status code unknown from: HEAD /
---> Elasticsearch.Net.PipelineException: Failed to ping the specified node.
---> Elasticsearch.Net.PipelineException: An error occurred trying to write the request data to the specified node.
---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid because of errors in the certificate chain: PartialChain
at System.Net.Security.SslStream.SendAuthResetSignal(ProtocolToken message, ExceptionDispatchInfo exception)
at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken).

One solution to fix this issue is disabling certificate validation on Elastic endpoint. To achieve this, you need to add another attribute to your Elastic target.

DisableCertificateValidation="true"

Now your Elastic target in nlog.config file should be something like this:

<target xsi:type="ElasticSearch" 
        name="elastic"               
        index="MyService-${date:format=yyyy.MM.dd}" 
        layout ="MyService-|${longdate}|${event-properties:item=EventId_Id}|${uppercase:${level}}|${logger}|${message} ${exception:format=tostring}|url: ${aspnet-request-url}|action: ${aspnet-mvc-action}" 
        includeAllProperties="true"
        requireAuth="true" 
        username="*******"
        password="*********" 
        uri="https://elasticSampleaddress.com:9200" />

That all. I hope you find this article is useful, let me know if you have any comment.

Read Writing logs into Elastic with NLog , ELK and .Net 5.0 (Part 1)

Follow me on Medium