New Relic

I needed to understand how to functionally use New Relic, the below are my notes I made in sequence while having a play. The accounts I created are free and I tried to use free tier AWS infastructure.

I found the new relic docs linked below really useful

• New Relic quick launch guide
• New Relic Explorer: View performance across apps, services, hosts

Create Account

This gives reporting, alerts and somewhere for your agent to push its logs to. You will need to get your licence key for the agent.

• https://newrelic.com/signup
• https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/

Agent configuration (Dockerfile)

You need to install an New Relic agent (this feeds data to your account). Using a base image like mcr.microsoft.com/dotnet/sdk:6.0 you can create your Dockerfile and install the agent manually

I built a working example and published the source to github.com/carlpaton/NewRelicHoon

This examples below are based on docs.newrelic.com and I have no idea what 548C16BF is or means… but it works :D/

# Install the agent
RUN apt-get update && apt-get install -y wget ca-certificates gnupg \
&& echo 'deb http://apt.newrelic.com/debian/ newrelic non-free' | tee /etc/apt/sources.list.d/newrelic.list \
&& wget https://download.newrelic.com/548C16BF.gpg \
&& apt-key add 548C16BF.gpg \
&& apt-get update \
&& apt-get install -y newrelic-dotnet-agent

Notes:

• CORECLR_PROFILER is not a secret, this is fom the New Relic .Net docs.
• The key shown below as INSERT_NEW_RELIC_LICENSE_KEY will look something like e0a8dd84aa0a8ddd444sdd24wdasdad3fsasd65edeNRAL and you can get it from your account.

# Enable the agent
ENV CORECLR_ENABLE_PROFILING=1 \
CORECLR_PROFILER={36032161-FFC0-4B61-B559-F6C5D41BAE5A} \
CORECLR_NEWRELIC_HOME=/usr/local/newrelic-dotnet-agent \
CORECLR_PROFILER_PATH=/usr/local/newrelic-dotnet-agent/libNewRelicProfiler.so \
NEW_RELIC_LICENSE_KEY=INSERT_NEW_RELIC_LICENSE_KEY \
NEW_RELIC_APP_NAME=INSERT_YOUR_APP_NAME

• https://docs.newrelic.com/install/dotnet/
• https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/linux-installation/install-infrastructure-monitoring-agent-linux/

These were some of the logs from the containers console

[1/Feb/2023:03:26:45] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:03:26:45] Connection id "0HMO0P632DR5U", Request id "0HMO0P632DR5U:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:03:26:45] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:03:26:45] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:03:26:45] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:03:26:45] --- End of stack trace from previous location ---
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:03:26:45] --- End of stack trace from previous location ---
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:26:45] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:03:31:39] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:03:31:39] Connection id "0HMO0P632DRBI", Request id "0HMO0P632DRBI:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:03:31:39] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:03:31:39] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:03:31:39] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:03:31:39] --- End of stack trace from previous location ---
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:03:31:39] --- End of stack trace from previous location ---
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:31:39] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:03:31:41] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:03:31:41] Connection id "0HMO0P632DRBI", Request id "0HMO0P632DRBI:00000005": An unhandled exception was thrown by the application.
[1/Feb/2023:03:31:41] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:03:31:41] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:03:31:41] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:03:31:41] --- End of stack trace from previous location ---
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:03:31:41] --- End of stack trace from previous location ---
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:31:41] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:03:56:47] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:03:56:47] Connection id "0HMO0P632DS89", Request id "0HMO0P632DS89:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:03:56:47] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:03:56:47] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:03:56:47] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:03:56:47] --- End of stack trace from previous location ---
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:03:56:47] --- End of stack trace from previous location ---
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:56:47] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:03:56:49] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:03:56:49] Connection id "0HMO0P632DS8B", Request id "0HMO0P632DS8B:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:03:56:49] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:03:56:49] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:03:56:49] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:03:56:49] --- End of stack trace from previous location ---
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:03:56:49] --- End of stack trace from previous location ---
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:56:49] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:04:01:39] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:04:01:39] Connection id "0HMO0P632DSDR", Request id "0HMO0P632DSDR:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:04:01:39] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:04:01:39] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:04:01:39] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:04:01:39] --- End of stack trace from previous location ---
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:04:01:39] --- End of stack trace from previous location ---
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:01:39] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:04:06:47] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:04:06:47] Connection id "0HMO0P632DSJM", Request id "0HMO0P632DSJM:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:04:06:47] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:04:06:47] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:04:06:47] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:04:06:47] --- End of stack trace from previous location ---
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:04:06:47] --- End of stack trace from previous location ---
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:06:47] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:04:21:39] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:04:21:39] Connection id "0HMO0P632DT4L", Request id "0HMO0P632DT4L:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:04:21:39] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:04:21:39] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:04:21:39] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:04:21:39] --- End of stack trace from previous location ---
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:04:21:39] --- End of stack trace from previous location ---
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:21:39] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

Synthetic Monitor

To get some data into the account (instead of running the URLs myself) I added some simple synthetic monitors. Pretty sure each have their usage but these would be the ones I would focuse on first if I needed some components monitored.

However the UI had several others:

• Synthetic monitoring best practices guide

Avalibility

Ping - Simple ping using cURL (avalibility ping endpoint)

For my tests I just used a ping monitor for both /ping and /health. I suspected them to fail 50% of the time as my code throws an error based on DateTime.Now.Second % 2 == 0 and my brain said there are 29 even numbers and 30 even numbers so surely it would be 50% :D

I was wrong and saw a higher success rate. I guess this is because I set the periods to 5 minutes and we cannot guarantee the second it will call. Meh too much math/statistics for this old brain.

User flow / functionality

Scripted browser - Full scripted browser test of user flow / functionality (page loads with user interactions)

• Introduction to scripted browser monitors

Endpoint availability

Scripted API - Uses an HTTP client to monitor remote API endpoints (REST, healthcheck endpoint, allows credentials etc).

You can add secrets using the Insert secure credential option in the UI, you then access them as $secure.CLIENT_SECRET. The authorisation service call is using OAuth2 Client Credentials Flow. Then see Import Node.js modules for help importing.

Example node 10 Runtime script

let request = require('request');
let assert = require('assert');

let authorisationService = {
    uri: 'https://authorisation-service.local/token',
    auth: {
        user: $secure.CLIENT_NAME,
        pass: $secure.CLIENT_SECRET,
    },
    formData: {
        grant_type: 'client_credentials'
    }
};

let healthCheck = function(err, response, token) {
    let tokenBody = JSON.parse(token);
    let options = {
        uri: 'https://newrelichoon.c4gqqjqcosehs.ap-southeast-2.cs.amazonlightsail.com/health',
        auth: {
            bearer: tokenBody.access_token
        }
    };

    request.get(options, function (err, response) {
      assert.equal(response.statusCode, 200);
  });
};

request.post(
    authorisationService,
    healthCheck
);

Query your data

I let the Synthetic Monitor above run for a weekend to get some data into events. There are event types (think of these as a table) and their dimensions of interest (think of these as columns in a table). Note that there are different data retention models for events based on your payment model, looks like for free accounts like mine its 8 days before my events will be deleted.

• Introduction to NRQL: the language of data
• Introduction to the metrics and events data explorer
• Introduction to the query builder
• New Relic data types: metrics, events, logs, and traces (MELT)

Log (event type)

“In New Relic, a log event refers to the information that is captured and stored in log statements generated by an application or system. Log events can contain a variety of information, such as messages, performance metrics, error messages, and debug information. In New Relic, log events can be captured from various sources, including log files, APIs, and agent-based integrations. These events are then stored in New Relic’s log management platform, where they can be searched, analyzed, and visualized for troubleshooting and performance monitoring purposes. The specific format and structure of log events in New Relic may vary depending on the logging framework or tool being used and the type of application or system being monitored.”

This didnt show me endpoints so I would guess its for application logs as a whole. The following are columns and their example data results I could see.

Example potentially useful query for dashboard / alert base on the below columns and their values.

SELECT count(*) FROM Log FACET `entity.name` WHERE level = 'ERROR' SINCE 30 MINUTES AGO TIMESERIES

• entity.name
  • This is the docker config name NEW_RELIC_APP_NAME=NewRelicHoon
• level
  • ERROR
• error.class
  • System.Exception (code)
• error.message
  • some sadness happend because your code is trash. Do better before ChatGPT replaces your ass. (code)

Span (event type)

“A “Span” event in New Relic is part of the Distributed Tracing feature offered by New Relic. Distributed tracing is a method of tracking the behavior of a complex, distributed system by recording information about the individual steps or segments of a transaction as it travels across different components of the system. A “Span” in this context represents a single, distinct segment of a transaction and the information recorded about it, such as its start and end times, its duration, and any associated metadata. Spans are usually generated by instrumenting the code of an application or by using a supported tracing library. The collected span events can be analyzed in New Relic to provide insights into the performance and behavior of a distributed system.” - Span events

SELECT count(*) FROM Span FACET `request.uri` SINCE 30 MINUTES AGO TIMESERIES
SELECT count(*) FROM Span FACET `response.status` SINCE 30 MINUTES AGO TIMESERIES

Summary of some dimensions (columns)

• request.method
  • GET
• request.uri
  • /, /health, /ping
• response.status
  • 404, 405, 500
• service.name
  • NewRelicHoon

SyntheticCheck (event type)

“SyntheticCheck returns metrics from one run of a specific monitor. These metrics include duration information for the monitor, location of the monitor check, size of the request and response headers, the type of monitor, and a timestamp..” - SyntheticCheck events

SELECT count(*) FROM SyntheticCheck SINCE 30 MINUTES AGO TIMESERIES

• typeLabel
  • Ping
• error
  • HTTPError: Server replied with a HTTP 500 response code

SyntheticRequest (event type)

“SyntheticRequest returns results from individual HTTP requests made during a check. The data gathered includes job information, location, type of content for request, duration information, request size, and page load information.” - SyntheticRequest events

SELECT count(*) FROM SyntheticRequest FACET path SINCE 30 MINUTES AGO TIMESERIES

• path
  • /health, /ping

Transation (event type)

“A “Transaction” event in New Relic is a high-level representation of a user request or a sequence of requests that are executed to complete a specific task. A transaction event in New Relic contains information about the start and end times of the transaction, the duration, and any associated metadata, such as the name of the transaction, the status code, and the URL.

In New Relic, transactions are generated by instrumenting the code of an application or by using a supported agent. The collected transaction events can be analyzed in New Relic to provide insights into the performance and behavior of an application, including the response time, error rate, and throughput.

By analyzing transaction events, developers and operations teams can identify bottlenecks, slowdowns, and other performance issues in their applications and infrastructure, and use the information to make informed decisions about how to optimize and improve their systems.”

SELECT count(*) FROM Transaction FACET `request.method` SINCE 30 MINUTES AGO TIMESERIES

• request.method
  • GET, HEAD
• request.uri
  • /, /ping, /health
• response.status
  • 404, 405, 500, 200
• error
  • true

TransationError (event type)

“A “TransactionError” event in New Relic is a type of transaction event that represents an error or exception that occurred during the execution of a transaction. A transaction error event contains information about the transaction that caused the error, such as the transaction name, the error message, the stack trace, and any associated metadata.

In New Relic, transaction error events are generated by instrumenting the code of an application or by using a supported agent. The collected transaction error events can be analyzed in New Relic to provide insights into the sources and causes of errors in an application.

By analyzing transaction error events, developers and operations teams can identify and troubleshoot errors in their applications, and use the information to make informed decisions about how to improve the reliability and stability of their systems.”

SELECT count(*) FROM TransactionError FACET `error.message` SINCE 30 MINUTES AGO TIMESERIES

• error.message
  • 405, 500, some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
• request.method
  • HEAD, GET
• request.uri
  • /ping, /health
• response.status
  • 405, 500
• transactionName
  • WebTransaction/StatusCode/405, WebTransaction/MVC/Ping/Get, WebTransaction/MVC/Ping/GetHealth
• transactionUiName
  • /405, /Ping/Get, /Ping/GetHealth

Alerts & AI

Application Performance Monitoring or APM (this allows for nrql queries to be run and reported on). This is useful for things like did the endpoint throw errors three times in last two hours - then alert!

1. Create Alert conditions (Policies). This can be used to split int groups like:

• Production
• UAT
• and other groups like PageDuty Alerts

2. Create subset of alerts that belong to the policy

• Here you would use the event queries (nrql) figured out in Query your data above to count events and ultimately alert on them.

Dashboards

The dashboard would be useful to see common sadness at a glance (potentially post a deployment). I simply added a test dashbord using the TransationError (event type) NRQL statment above to peek at 405 and 500 errors over a period of time.

• Introduction to dashboards

Terraform

Terraform would be the preferred way to create New Relic infastructure for Synthetic Monitoring and Alerts & AI - you always want to be able to commit a change, get review and publish a repeatable step to your environments.

I didnt see the point in copying the doc examples so instead link to them below and noted the suggested resource name from docs.

Alerts & AI

newrelic_nrql_alert_condition

• https://registry.terraform.io/providers/newrelic/newrelic/latest/docs/resources/nrql_alert_condition
• https://registry.terraform.io/providers/newrelic/newrelic/latest/docs/resources/infra_alert_condition

Synthetic Monitor

newrelic_alert_condition
newrelic_synthetics_alert_condition

• https://registry.terraform.io/providers/newrelic/newrelic/latest/docs/resources/synthetics_alert_condition