Real-time serverless network packet capture for instant debugging

When debugging distributed systems—whether APIs, microservices, or cloud-native apps—developers typically rely on request logs stored in services like CloudWatch. These logs capture processed data, which is often sufficient for routine issues. However, some problems defy explanation through high-level logs alone.

For instance, imagine your application adds a critical security header to a response, but the client never applies it. Is the client malfunctioning? Did a proxy strip the header? Or did the client never receive the response at all? Standard request logs won’t answer these questions, because they only show the transformed data after processing. To uncover the root cause, you need visibility into the raw network traffic as it travels between systems.

This is where packet capture becomes essential. Tools like Wireshark have long provided local network visibility, even supporting remote captures via SSH. But sharing these captures in real time with distributed teams presents challenges. Uploading .pcap files to Slack or emailing them is cumbersome, and screen-sharing during calls forces collaborators to navigate the data indirectly. A better approach is needed—one that delivers live, shareable network traffic views without the friction.

How real-time packet sharing accelerates debugging

A new serverless solution leverages AWS AppSync Events to stream packet captures in real time to any collaborator with a link. This eliminates the need to manually exchange files or rely on indirect screen sharing. The system is designed for teams working on custom UDP APIs or distributed systems where network-level issues require immediate, collective inspection.

The architecture is intentionally simple, combining AWS services to create a seamless pipeline:

• Packets arriving at a UDP listener are queued in Amazon SQS for processing.
• A Lambda function parses IP and UDP headers, extracts relevant metadata, and formats the data as JSON.
• The processed events are published to an AppSync channel, which streams them in real time to a web-based UI.
• A second Lambda function serves the UI via a function URL, enabling instant access without managing servers.

This setup requires no persistent infrastructure—only a one-time deployment. Once live, any team member can open a browser, access the link, and observe incoming network traffic as it happens.

Deploying the solution with minimal setup

The infrastructure is defined using AWS CloudFormation, making deployment straightforward. The template creates three core components:

• An AppSync API with API key authentication for secure access.
• An API key to authenticate requests from the UI and Lambda functions.
• A channel namespace named packets, where all captured events are published.

Here’s a simplified excerpt of the CloudFormation template:

PacketCaptureApi:
  Type: AWS::AppSync::Api
  Properties:
    Name: !Sub "${AWS::StackName}-packet-capture"
    EventConfig:
      AuthProviders:
        - AuthType: API_KEY
      ConnectionAuthModes:
        - AuthType: API_KEY
      DefaultPublishAuthModes:
        - AuthType: API_KEY
      DefaultSubscribeAuthModes:
        - AuthType: API_KEY

PacketCaptureApiKey:
  Type: AWS::AppSync::ApiKey
  Properties:
    ApiId: !GetAtt PacketCaptureApi.ApiId

PacketsNamespace:
  Type: AWS::AppSync::ChannelNamespace
  Properties:
    ApiId: !GetAtt PacketCaptureApi.ApiId
    Name: packets

The Lambda function responsible for publishing events is equally concise. It constructs a JSON payload containing the parsed packet data and sends it to the AppSync channel via an HTTP POST request. The function includes error handling to log failures and ensure reliability.

private static async Task PublishEventsAsync(PacketEvent[] events, ILambdaContext context)
{
  var body = new PublishBody
  {
    Channel = APPSYNC_CHANNEL,
    Events = events.Select(e => JsonSerializer.Serialize(e, JsonContext.Default.PacketEvent)).ToList()
  };

  var json = JsonSerializer.Serialize(body, JsonContext.Default.PublishBody);
  using var request = new HttpRequestMessage(HttpMethod.Post, APPSYNC_HTTP_URL)
  {
    Content = new StringContent(json, Encoding.UTF8, "application/json")
  };

  request.Headers.Add("x-api-key", APPSYNC_API_KEY);
  var response = await Http.SendAsync(request);

  if (!response.IsSuccessStatusCode)
  {
    var error = await response.Content.ReadAsStringAsync();
    context.Logger.LogError($"AppSync publish failed ({response.StatusCode}): {error}");
    response.EnsureSuccessStatusCode();
  }
}

Full code and a step-by-step video walkthrough are available in the GitHub repository. The guide also includes instructions for setting up a WireGuard tunnel to encrypt sensitive data during transit, ensuring privacy for production environments.

Why this matters for modern debugging

Packet-level debugging has traditionally required local tools or complex setups, limiting collaboration and slowing down incident response. By combining serverless AWS services—AppSync, SQS, Lambda, and UDP Gateway—this solution delivers a real-time, multi-user packet capture experience that was previously difficult to achieve.

For teams building APIs, IoT systems, or any networked application, this approach transforms how issues are diagnosed. Instead of waiting for logs or exchanging files, engineers can now observe and discuss network behavior in real time, reducing downtime and improving system reliability.

The future of software development belongs to tools that are not only powerful but also frictionless. Real-time packet capture exemplifies this trend—simple to deploy, instant to use, and built for collaboration in a distributed world.