New Relic

New Relic Hoon

I needed to understand how to functionally use New Relic, the below are my notes I made in sequence while having a play. The accounts I created are free and I tried to use free tier AWS infastructure.

I found the new relic docs linked below really useful

Create Account

This gives reporting, alerts and somewhere for your agent to push its logs to. You will need to get your licence key for the agent.

Agent configuration (Dockerfile)

You need to install an New Relic agent (this feeds data to your account). Using a base image like mcr.microsoft.com/dotnet/sdk:6.0 you can create your Dockerfile and install the agent manually

I built a working example and published the source to github.com/carlpaton/NewRelicHoon

This examples below are based on docs.newrelic.com and I have no idea what 548C16BF is or means… but it works :D/

1
2
3
4
5
6
7
# Install the agent
RUN apt-get update && apt-get install -y wget ca-certificates gnupg \
&& echo 'deb http://apt.newrelic.com/debian/ newrelic non-free' | tee /etc/apt/sources.list.d/newrelic.list \
&& wget https://download.newrelic.com/548C16BF.gpg \
&& apt-key add 548C16BF.gpg \
&& apt-get update \
&& apt-get install -y newrelic-dotnet-agent

Notes:

  • CORECLR_PROFILER is not a secret, this is fom the New Relic .Net docs.
  • The key shown below as INSERT_NEW_RELIC_LICENSE_KEY will look something like e0a8dd84aa0a8ddd444sdd24wdasdad3fsasd65edeNRAL and you can get it from your account.
1
2
3
4
5
6
7
# Enable the agent
ENV CORECLR_ENABLE_PROFILING=1 \
CORECLR_PROFILER={36032161-FFC0-4B61-B559-F6C5D41BAE5A} \
CORECLR_NEWRELIC_HOME=/usr/local/newrelic-dotnet-agent \
CORECLR_PROFILER_PATH=/usr/local/newrelic-dotnet-agent/libNewRelicProfiler.so \
NEW_RELIC_LICENSE_KEY=INSERT_NEW_RELIC_LICENSE_KEY \
NEW_RELIC_APP_NAME=INSERT_YOUR_APP_NAME
These were some of the logs from the containers console
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
[1/Feb/2023:03:26:45] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:03:26:45] Connection id "0HMO0P632DR5U", Request id "0HMO0P632DR5U:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:03:26:45] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:03:26:45] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:03:26:45] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:03:26:45] --- End of stack trace from previous location ---
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:03:26:45] --- End of stack trace from previous location ---
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:26:45] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:26:45] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:03:31:39] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:03:31:39] Connection id "0HMO0P632DRBI", Request id "0HMO0P632DRBI:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:03:31:39] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:03:31:39] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:03:31:39] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:03:31:39] --- End of stack trace from previous location ---
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:03:31:39] --- End of stack trace from previous location ---
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:31:39] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:31:39] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:03:31:41] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:03:31:41] Connection id "0HMO0P632DRBI", Request id "0HMO0P632DRBI:00000005": An unhandled exception was thrown by the application.
[1/Feb/2023:03:31:41] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:03:31:41] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:03:31:41] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:03:31:41] --- End of stack trace from previous location ---
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:03:31:41] --- End of stack trace from previous location ---
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:31:41] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:31:41] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:03:56:47] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:03:56:47] Connection id "0HMO0P632DS89", Request id "0HMO0P632DS89:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:03:56:47] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:03:56:47] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:03:56:47] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:03:56:47] --- End of stack trace from previous location ---
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:03:56:47] --- End of stack trace from previous location ---
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:56:47] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:56:47] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:03:56:49] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:03:56:49] Connection id "0HMO0P632DS8B", Request id "0HMO0P632DS8B:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:03:56:49] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:03:56:49] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:03:56:49] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:03:56:49] --- End of stack trace from previous location ---
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:03:56:49] --- End of stack trace from previous location ---
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:56:49] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:03:56:49] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:04:01:39] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:04:01:39] Connection id "0HMO0P632DSDR", Request id "0HMO0P632DSDR:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:04:01:39] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:04:01:39] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:04:01:39] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:04:01:39] --- End of stack trace from previous location ---
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:04:01:39] --- End of stack trace from previous location ---
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:01:39] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:01:39] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:04:06:47] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:04:06:47] Connection id "0HMO0P632DSJM", Request id "0HMO0P632DSJM:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:04:06:47] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:04:06:47] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:04:06:47] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:04:06:47] --- End of stack trace from previous location ---
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:04:06:47] --- End of stack trace from previous location ---
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:06:47] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:06:47] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[1/Feb/2023:04:21:39] fail: Microsoft.AspNetCore.Server.Kestrel[13]
[1/Feb/2023:04:21:39] Connection id "0HMO0P632DT4L", Request id "0HMO0P632DT4L:00000003": An unhandled exception was thrown by the application.
[1/Feb/2023:04:21:39] System.Exception: some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
[1/Feb/2023:04:21:39] at NewRelicHoon.Controllers.PingController.Get() in /src/Controllers/PingController.cs:line 16
[1/Feb/2023:04:21:39] at lambda_method467(Closure , Object , Object[] )
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
[1/Feb/2023:04:21:39] --- End of stack trace from previous location ---
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
[1/Feb/2023:04:21:39] --- End of stack trace from previous location ---
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:21:39] at NewRelic.Providers.Wrapper.AspNetCore.WrapPipelineMiddleware.Invoke(HttpContext context)
[1/Feb/2023:04:21:39] at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

Synthetic Monitor

To get some data into the account (instead of running the URLs myself) I added some simple synthetic monitors. Pretty sure each have their usage but these would be the ones I would focuse on first if I needed some components monitored.

However the UI had several others:

Synthetic Monitoring Type

Avalibility

Ping - Simple ping using cURL (avalibility ping endpoint)

For my tests I just used a ping monitor for both /ping and /health. I suspected them to fail 50% of the time as my code throws an error based on DateTime.Now.Second % 2 == 0 and my brain said there are 29 even numbers and 30 even numbers so surely it would be 50% :D

I was wrong and saw a higher success rate. I guess this is because I set the periods to 5 minutes and we cannot guarantee the second it will call. Meh too much math/statistics for this old brain.

Synthetic Monitoring

User flow / functionality

Scripted browser - Full scripted browser test of user flow / functionality (page loads with user interactions)

Endpoint availability

Scripted API - Uses an HTTP client to monitor remote API endpoints (REST, healthcheck endpoint, allows credentials etc).

You can add secrets using the Insert secure credential option in the UI, you then access them as $secure.CLIENT_SECRET. The authorisation service call is using OAuth2 Client Credentials Flow. Then see Import Node.js modules for help importing.

Example node 10 Runtime script
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
let request = require('request');
let assert = require('assert');

let authorisationService = {
uri: 'https://authorisation-service.local/token',
auth: {
user: $secure.CLIENT_NAME,
pass: $secure.CLIENT_SECRET,
},
formData: {
grant_type: 'client_credentials'
}
};

let healthCheck = function(err, response, token) {
let tokenBody = JSON.parse(token);
let options = {
uri: 'https://newrelichoon.c4gqqjqcosehs.ap-southeast-2.cs.amazonlightsail.com/health',
auth: {
bearer: tokenBody.access_token
}
};

request.get(options, function (err, response) {
assert.equal(response.statusCode, 200);
});
};

request.post(
authorisationService,
healthCheck
);

Query your data

I let the Synthetic Monitor above run for a weekend to get some data into events. There are event types (think of these as a table) and their dimensions of interest (think of these as columns in a table). Note that there are different data retention models for events based on your payment model, looks like for free accounts like mine its 8 days before my events will be deleted.

Log (event type)

“In New Relic, a log event refers to the information that is captured and stored in log statements generated by an application or system. Log events can contain a variety of information, such as messages, performance metrics, error messages, and debug information. In New Relic, log events can be captured from various sources, including log files, APIs, and agent-based integrations. These events are then stored in New Relic’s log management platform, where they can be searched, analyzed, and visualized for troubleshooting and performance monitoring purposes. The specific format and structure of log events in New Relic may vary depending on the logging framework or tool being used and the type of application or system being monitored.”

This didnt show me endpoints so I would guess its for application logs as a whole. The following are columns and their example data results I could see.

Example potentially useful query for dashboard / alert base on the below columns and their values.

1
SELECT count(*) FROM Log FACET `entity.name` WHERE level = 'ERROR' SINCE 30 MINUTES AGO TIMESERIES
  • entity.name
  • level
    • ERROR
  • error.class
    • System.Exception (code)
  • error.message
    • some sadness happend because your code is trash. Do better before ChatGPT replaces your ass. (code)

Log Event type

Span (event type)

“A “Span” event in New Relic is part of the Distributed Tracing feature offered by New Relic. Distributed tracing is a method of tracking the behavior of a complex, distributed system by recording information about the individual steps or segments of a transaction as it travels across different components of the system. A “Span” in this context represents a single, distinct segment of a transaction and the information recorded about it, such as its start and end times, its duration, and any associated metadata. Spans are usually generated by instrumenting the code of an application or by using a supported tracing library. The collected span events can be analyzed in New Relic to provide insights into the performance and behavior of a distributed system.” - Span events

1
2
SELECT count(*) FROM Span FACET `request.uri` SINCE 30 MINUTES AGO TIMESERIES
SELECT count(*) FROM Span FACET `response.status` SINCE 30 MINUTES AGO TIMESERIES

Summary of some dimensions (columns)

  • request.method
    • GET
  • request.uri
    • /, /health, /ping
  • response.status
    • 404, 405, 500
  • service.name
    • NewRelicHoon

Span Event type

SyntheticCheck (event type)

“SyntheticCheck returns metrics from one run of a specific monitor. These metrics include duration information for the monitor, location of the monitor check, size of the request and response headers, the type of monitor, and a timestamp..” - SyntheticCheck events

1
SELECT count(*) FROM SyntheticCheck SINCE 30 MINUTES AGO TIMESERIES
  • typeLabel
    • Ping
  • error
    • HTTPError: Server replied with a HTTP 500 response code

Synthetic Check Event type

SyntheticRequest (event type)

“SyntheticRequest returns results from individual HTTP requests made during a check. The data gathered includes job information, location, type of content for request, duration information, request size, and page load information.” - SyntheticRequest events

1
SELECT count(*) FROM SyntheticRequest FACET path SINCE 30 MINUTES AGO TIMESERIES
  • path
    • /health, /ping

Synthetic Request Event type

Transation (event type)

“A “Transaction” event in New Relic is a high-level representation of a user request or a sequence of requests that are executed to complete a specific task. A transaction event in New Relic contains information about the start and end times of the transaction, the duration, and any associated metadata, such as the name of the transaction, the status code, and the URL.

In New Relic, transactions are generated by instrumenting the code of an application or by using a supported agent. The collected transaction events can be analyzed in New Relic to provide insights into the performance and behavior of an application, including the response time, error rate, and throughput.

By analyzing transaction events, developers and operations teams can identify bottlenecks, slowdowns, and other performance issues in their applications and infrastructure, and use the information to make informed decisions about how to optimize and improve their systems.”

1
SELECT count(*) FROM Transaction FACET `request.method` SINCE 30 MINUTES AGO TIMESERIES
  • request.method
    • GET, HEAD
  • request.uri
    • /, /ping, /health
  • response.status
    • 404, 405, 500, 200
  • error
    • true

Transation Event type

TransationError (event type)

“A “TransactionError” event in New Relic is a type of transaction event that represents an error or exception that occurred during the execution of a transaction. A transaction error event contains information about the transaction that caused the error, such as the transaction name, the error message, the stack trace, and any associated metadata.

In New Relic, transaction error events are generated by instrumenting the code of an application or by using a supported agent. The collected transaction error events can be analyzed in New Relic to provide insights into the sources and causes of errors in an application.

By analyzing transaction error events, developers and operations teams can identify and troubleshoot errors in their applications, and use the information to make informed decisions about how to improve the reliability and stability of their systems.”

1
SELECT count(*) FROM TransactionError FACET `error.message` SINCE 30 MINUTES AGO TIMESERIES
  • error.message
    • 405, 500, some sadness happend because your code is trash. Do better before ChatGPT replaces your ass.
  • request.method
    • HEAD, GET
  • request.uri
    • /ping, /health
  • response.status
    • 405, 500
  • transactionName
    • WebTransaction/StatusCode/405, WebTransaction/MVC/Ping/Get, WebTransaction/MVC/Ping/GetHealth
  • transactionUiName
    • /405, /Ping/Get, /Ping/GetHealth

Transation Error Event type

Alerts & AI

Application Performance Monitoring or APM (this allows for nrql queries to be run and reported on). This is useful for things like did the endpoint throw errors three times in last two hours - then alert!

  1. Create Alert conditions (Policies). This can be used to split int groups like:
  • Production
  • UAT
  • and other groups like PageDuty Alerts
  1. Create subset of alerts that belong to the policy
  • Here you would use the event queries (nrql) figured out in Query your data above to count events and ultimately alert on them.

Dashboards

The dashboard would be useful to see common sadness at a glance (potentially post a deployment). I simply added a test dashbord using the TransationError (event type) NRQL statment above to peek at 405 and 500 errors over a period of time.

Test Dashboard

Terraform

Terraform would be the preferred way to create New Relic infastructure for Synthetic Monitoring and Alerts & AI - you always want to be able to commit a change, get review and publish a repeatable step to your environments.

I didnt see the point in copying the doc examples so instead link to them below and noted the suggested resource name from docs.

Alerts & AI

1
newrelic_nrql_alert_condition

Synthetic Monitor

1
2
newrelic_alert_condition
newrelic_synthetics_alert_condition