Architecture

Emp-Mix is designed to handle high-throughput event ingestion with zero data loss. This page outlines how data flows from your application to our databases.

The Event Lifecycle

1. Client-Side Generation

A user clicks a "Buy Now" button. Your frontend app calls EmpMix.track("Purchase"). The SDK appends metadata like Browser, OS, Timestamp, and the current distinct_id (either anonymous or authenticated).

The SDK checks its flush interval and batch size. When conditions are met, it sends the batch payload to the Emp-Mix ingest server.

2. Ingestion (empmix-backend-api)

The backend API receives the POST request. It validates the apiKey. Instead of writing directly to the database (which can cause bottlenecks), it converts the payload into a message and publishes it to a RabbitMQ Exchange.

It immediately returns a 200 OK response. The entire HTTP request finishes in milliseconds.

3. Message Broker (RabbitMQ)

RabbitMQ places the message into the events_queue. It guarantees message durability and allows for buffering during traffic spikes (e.g., Black Friday sales).

4. Consumer Processing (empmix-event-consumer)

A separate Node.js worker constantly listens to the queue. When it pulls the message:

  1. It looks up the distinct_id to resolve identity.
  2. It updates session metrics (last active time, session duration).
  3. It formats the event object and performs an upsert/bulk write to MongoDB.

5. Storage & Analytics (MongoDB)

The event is permanently stored in the MongoDB events collection. When a user opens the Customer Dashboard, the frontend queries the private API. The backend translates the query builder state into a complex MongoDB Aggregation Pipeline, retrieving the newly inserted event to visualize on the chart.

Architecture Diagram

graph TD
    Client[Client SDK (Browser/Mobile)] -->|POST /track| Ingest[Backend API]
    Ingest -->|Publish| RabbitMQ[(RabbitMQ)]
    RabbitMQ -->|Consume| Worker[Event Consumer]
    Worker -->|Write| MongoDB[(MongoDB)]
    Dashboard[Customer Dashboard] -->|Query| MongoDB
Last updated on June 4, 2026