Four common methods for generating distributed IDs

I won't go into detail about why this problem arises—a single database has limited capacity, and once you start sharding databases and tables, ensuring unique IDs becomes a problem that needs to be solved.

Requirements:

1. Uniqueness: At any time, no node is allowed to generate duplicate IDs.
2. High performance: Must generate IDs quickly under high concurrency without becoming a performance bottleneck.
3. Security: Users must not be able to guess the ID pattern, to avoid security risks.

Snowflake Algorithm

   +---------------------+       +---------------------+
   |   Node A (Worker 1) | <---> |   Clock + Counter   |
   +---------------------+       +---------------------+
             |
             |
             |--- ID1: 64-bit
             |    0      00001     00000001     000000000000
             |   |        |            |               |
             |   |        |            |               +-- Sequence (12 bits): counter within the same millisecond
             |   |        |            +------------------ Machine ID (10 bits)
             |   |        +------------------------------ Data Center ID (5 bits)
             |   +---------------------------------------- Timestamp (41 bits)
             v
     ID: 64-bit integer, generated locally

Generation logic:

• Each node is configured with a unique machine ID and data center ID.
• Get the current timestamp, then concatenate the data center ID, machine ID, and sequence number.
• If the sequence overflows within the same millisecond, wait for the next millisecond.

How uniqueness is ensured across multiple nodes: Uniqueness is guaranteed by the combination of different machine IDs and timestamps.

Characteristics: Generated locally, no central node required, high performance and low latency.

Redis

   +-----------+         +-------------+
   | Node A    | ----+-> |             |
   |           |     |   |             |
   +-----------+     |   |             |
                     |   |  Redis Master|
   +-----------+     |   |   INCR key  |
   | Node B    | ----+-> |             |
   +-----------+         |             |
                         +-------------+
                               |
                               v
                        ID1: 10000000001
                        ID2: 10000000002

Generation logic:

• Each node obtains an auto-incrementing integer via INCR my_id_key.
• Redis guarantees atomicity—each increment by 1 returns a unique ID.

How uniqueness is ensured across multiple nodes: All nodes access the same Redis instance, and Redis guarantees global uniqueness.

Characteristics: Centralized generation, simple and reliable, but depends on Redis's central performance.

Single MySQL Instance

   +-----------+         +-------------------+
   | Node A    | ----+-> |                   |
   +-----------+     |   |  MySQL Single     |
                     |   |  Instance         |
   +-----------+     |   |  INSERT to get    |
   | Node B    | ----+-> |  auto-increment ID|
   +-----------+         | (auto_increment)  |
                         +-------------------+
                               |
                               v
                          ID1: 1
                          ID2: 2

Generation logic:

• All nodes insert records into the same MySQL table.
• Use the AUTO_INCREMENT field to automatically generate IDs.
• Can use select last_insert_id() to retrieve the ID from the current insert.

How uniqueness is ensured across multiple nodes: All nodes rely on the same database instance, and the global auto-increment field ensures uniqueness.

Characteristics: Simple but not suitable for high concurrency; depends on database performance.

MySQL Segment Mode

        +----------------------+
        |   MySQL Segment Table|
        |----------------------|
        | biz_tag | max_id     |
        | order   | 10500      |
        +----------------------+
               ^          |
               | Retrieve a segment (10001 ~ 10500)
               |
   +-----------+-----------+
   |                       |
+--------+         +--------+
| Node A |         | Node B |
+--------+         +--------+
   |                    |
   | Local increment:   | Local increment:
   | 10001++            | 10501++
   v                    v
ID1: 10001          ID2: 10501

Generation logic:

• Each node requests a segment of IDs from MySQL (e.g., 1000 at a time).
• The MySQL table maintains the maximum ID for each business.
• Once a node receives a segment, it caches and uses it locally.

How uniqueness is ensured across multiple nodes:

• Nodes do not communicate with each other; the database segment table allocates non-overlapping ranges.

Characteristics: Centralized coordination, local generation on nodes, good performance, suitable for high concurrency.