I have never racked a server. I have opened a lot of tickets asking someone else to.

That is the normal shape of this job now. You are a competent Linux engineer, you own machines that live in a building you have visited maybe twice, and every physical act on those machines happens through a stranger's hands and a ticketing system. It works fine right up until the moment someone says "the B-side feed is down on the cabinet, can you confirm your dual-corded gear is holding" and you realise you are nodding at words rather than understanding them.

This series is me closing that gap, written down. It is not a datacenter design course — I am not going to teach you to size a CRAC unit or argue about Tier ratings, because neither of us is ever going to do that. It is the subset a Linux engineer needs in order to describe their own hardware accurately, read a facility's paperwork, and write a remote hands ticket that gets the job done on the first attempt.

Part 0 is vocabulary. If you already know what a demarc is, skip to Part 1.

The five nested boxes

Almost every term in this series is a location. The whole model is five boxes inside each other, and knowing which box a word belongs to is most of the battle.

┌─ FACILITY ────────────────────────────────────────────────┐
│  the building. Has an address, a loading dock, a security  │
│  desk, generators, chillers, and a Meet-Me Room.           │
│                                                            │
│  ┌─ DATA HALL ─────────────────────────────────────────┐  │
│  │  one room full of rows. Also called a "suite".      │  │
│  │                                                      │  │
│  │  ┌─ CAGE ────────────────────────────────────────┐  │  │
│  │  │  your fenced-off floor space. Locked. Yours.  │  │  │
│  │  │  (If you rent single cabinets instead, you    │  │  │
│  │  │   have no cage — skip this box.)              │  │  │
│  │  │                                                │  │  │
│  │  │  ┌─ RACK / CABINET ────────────────────────┐  │  │  │
│  │  │  │  the 19-inch steel frame. 42U-48U tall. │  │  │  │
│  │  │  │                                          │  │  │  │
│  │  │  │  ┌─ SERVER ──────────────────────────┐  │  │  │  │
│  │  │  │  │  the thing you actually ssh into. │  │  │  │  │
│  │  │  │  └───────────────────────────────────┘  │  │  │  │
│  │  │  └──────────────────────────────────────────┘  │  │  │
│  │  └────────────────────────────────────────────────┘  │  │
│  └──────────────────────────────────────────────────────┘  │
└────────────────────────────────────────────────────────────┘

Your ssh session reaches the innermost box. Everything else in this series is one of the outer four, and every one of them can take your service down without a single thing being wrong inside Linux.

Rack and cabinet

Used interchangeably in conversation. Strictly, a rack is an open frame — four posts, no sides — and a cabinet is an enclosed rack with doors and side panels. Colocation is almost always cabinets, because the doors are lockable and the enclosure is what makes front-to-back airflow work.

The number you will see everywhere is U (or RU, rack unit): the vertical unit of mounting space, standardised at 1.75 inches / 44.45 mm. A "1U server" is one unit tall, a "2U server" is two. A typical colo cabinet is 42U or 48U. Part 1 pulls this apart properly.

Front and back

This is the one that surprised me by how much it matters, and it is the answer to "what do I expect at the front versus the back of the rack".

The front is where the cold air is and where nothing is plugged in. Front of a server: bezel, drive bays, power button, status LEDs, a service tag, sometimes a small LCD or a USB/serial port for a technician. That is essentially it.

The back is where every cable is. Network, power, out-of-band management, console. All of it.

        COLD AISLE                              HOT AISLE
   (air conditioning in)                    (hot exhaust out)
            │                                       │
            v                                       v
    ┌───────────────────┐                   ┌───────────────────┐
    │  FRONT of rack    │                   │   REAR of rack    │
    │                   │  ═══ airflow ══>  │                   │
    │  · drive bays     │                   │  · NIC ports      │
    │  · power button   │                   │  · BMC/IPMI port  │
    │  · status LEDs    │                   │  · PSU inlets     │
    │  · service tag    │                   │  · PDUs (A and B) │
    │  · blanking panels│                   │  · patch panels   │
    └───────────────────┘                   └───────────────────┘

Servers pull cold air in the front and blow hot air out the back. Rows are arranged so that all the fronts face each other across one aisle (the cold aisle) and all the backs face each other across the next (the hot aisle). That is hot aisle / cold aisle containment, and it is why you cannot simply turn a cabinet around to make cabling easier.

Practical consequence for you: almost every remote hands task happens at the back. If your ticket says "the port on the left" without saying whether you mean facing the front or the back, you have written an ambiguous ticket. Say rear, and say left-or-right as viewed from the rear.

A-side and B-side

Your cabinet gets two independent power feeds, traced back through separate breakers, separate distribution panels, and ideally separate UPS systems. They are called the A-side and the B-side. Each feeds its own PDU (power distribution unit — the metal strip of outlets running down the inside of the cabinet).

A server with two power supplies has one cord in the A PDU and one in the B PDU. Lose an entire side — a tripped breaker, a UPS in maintenance — and the server keeps running on the other.

Two things I did not appreciate until I went looking. First, this only protects you if the machine genuinely has two PSUs and both are genuinely plugged into different sides; a single-PSU box in a dual-fed cabinet has no redundancy at all, and it is depressingly easy to have one cabinet in which somebody cabled both cords of a server into the same PDU. Second, redundancy costs you capacity: if either side must be able to carry the whole cabinet alone, neither side can normally run above roughly half its usable rating. Part 2 does the arithmetic, including the 80 % rule that makes "usable" smaller than you think.

Demarc

The demarcation point, universally shortened to demarc, is the boundary of responsibility. On one side, the facility's or carrier's equipment and problem. On the other, yours.

Physically it is normally unglamorous: a patch panel, in or near your cage, where the facility's cabling ends and you are allowed to plug in. When a circuit breaks, the first question anyone asks is which side of the demarc the fault is on, because that decides whose job it is to fix.

The word comes from telephony, where the demarc was the box on the outside of the building. Same idea, indoor version.

Meet-Me Room and cross-connect

The Meet-Me Room (MMR) is a secure room in the facility where all the carriers, cloud on-ramps and other tenants terminate their networks. Nobody gets casual access to it. It is the building's switchboard.

A cross-connect is a physical cable the facility installs for you between two points in the building — typically from your cage's demarc panel to some other party's termination in the MMR. It is not a service or a circuit. It is literally a piece of fibre or copper that a technician runs and patches, which is why it carries an install fee and a monthly rental and why it takes days rather than seconds.

To order one you supply an LOA-CFA: Letter of Authorization / Connecting Facility Assignment. The other party issues it, and it is the document that says "this customer is permitted to connect to us, at this exact cabinet, panel and port." Without it the facility will not patch anything, because they have no proof the far end consented. AWS, for example, issues an LOA-CFA for a Direct Connect port and it expires if the cross-connect is not completed in time.

Two more terms that ride along: the A-side and Z-side of a circuit. Confusingly this is not the A/B of power — here A is simply one end of the circuit and Z is the other. Context tells you which meaning is in play: power has A and B, circuits have A and Z.

Direct connect

A direct connection to a cloud provider is a cross-connect whose far end happens to be AWS, Azure or Google rather than a carrier. The commercial names are AWS Direct Connect, Azure ExpressRoute and Google Cloud Interconnect. Traffic to that cloud leaves your cabinet, crosses the MMR on your fibre, and lands on the provider's router without ever touching the public internet.

You will meet two flavours. A dedicated connection is a whole physical port that is yours, in AWS's case 1, 10, 100 or 400 Gbps, and you order the cross-connect yourself. A hosted connection is a slice of somebody else's port sold to you by a partner, available at much smaller sizes, with no cross-connect for you to order because the partner already has one. Part 6 covers which you actually want.

DAC, AOC, transceivers

A DAC is a Direct Attach Copper cable: a fixed length of twinaxial copper with the transceiver moulded onto both ends. You cannot unplug the ends. You buy it as one unit — "a 3-metre SFP28 DAC" — and it is the cheap, low-power, low-latency way to connect a server to a switch in the same rack. Cheap and reliable, but short: passive DACs are typically good to about 3 m and stretch to 5 or 7 m; active ones with signal conditioning built in reach roughly 10 m.

An AOC is an Active Optical Cable: same idea, ends moulded on, but the middle is fibre and the ends contain lasers. Much longer reach — up to about 100 m — for more money and more power.

A transceiver (a module, an "optic", an SFP) is the removable part on its own: you plug the transceiver into the switch and a separate fibre patch lead into the transceiver. That is what you use when the run is long, goes through a patch panel, or crosses to another rack.

The form factor names encode speed. SFP+ is 10 Gb, SFP28 is 25 Gb, QSFP+ is 40 Gb, QSFP28 is 100 Gb. Part 3 covers the whole family, plus which fibre goes with which and why the colour of the cable tells you something real.

IPMI, BMC, iDRAC, iLO

Your server contains a second, tiny computer that runs whether or not the main machine is powered on. It is called the BMC — Baseboard Management Controller. It has its own processor, its own firmware, its own network stack, and its own IP address, and it can power the server on and off, show you the console, mount an ISO, and read every temperature and fan in the chassis.

IPMI is the old standard protocol for talking to it. Redfish is the modern HTTPS/JSON replacement. The vendors each brand their BMC differently, which is why the same thing has four names: Dell calls it iDRAC, HPE calls it iLO, Lenovo XCC, Supermicro just IPMI.

The cable question you asked about: on the back of the server there is usually a small RJ45 labelled BMC, IPMI, iDRAC or iLO, physically separate from the data NICs. That is the dedicated management port and it is the one you want. The alternative is shared mode, where the BMC borrows one of the main NIC ports over a sideband channel called NC-SI. Shared saves you a switch port and a cable; it also means that whatever takes out the main NIC takes out your way of fixing the main NIC. Part 4 argues that one out properly, along with why a BMC must never be reachable from the internet.

Remote hands and smart hands

Remote hands is the basic, usually contract-included tier: a technician follows precise instructions. Press this button. Reseat that cable. Read me the LEDs. Tell me what the LCD says. They are not expected to diagnose anything, and you should not expect them to.

Smart hands is the chargeable tier for work that needs judgement or real hardware skill: racking and cabling a new server, replacing a PSU or a disk, receiving and unboxing a shipment, testing a circuit.

The distinction matters commercially, but the operational lesson is the same for both: the technician cannot see your monitoring, does not know your naming scheme, and is standing in a loud cold room in front of forty identical black boxes. Every ambiguity in your ticket becomes either a phone call or a mistake. Part 7 is entirely about writing tickets that survive that environment.

What is coming

  • Part 1 — the rack itself: U numbering, rails, cage nuts, depth, weight, blanking panels, and how to read a rack elevation.
  • Part 2 — power: A/B feeds, PDU types, C13 through C20, the 80 % rule, and why redundancy halves your capacity.
  • Part 3 — cables and optics: DAC, AOC, transceivers, multimode versus singlemode, LC and MPO, and picking a length.
  • Part 4 — out-of-band: BMC, IPMI, Redfish, serial consoles, switched PDUs, and the three different things "reboot it" can mean.
  • Part 5 — where the switches live: top-of-rack versus end-of-row, leaf-spine, uplinks, oversubscription.
  • Part 6 — leaving the cage: demarc, MMR, cross-connects, direct connects, transit, peering and IXs.
  • Part 7 — directing remote hands: the ticket template, the standard jobs, and how to verify the work.
  • Part 8 — the cheat sheet: every number and connector in this series on one page.

A note on how this was written, since the blog's rule is not to claim experience I do not have: this series is built from vendor and standards documentation, facility operator guidance, and the parts of the job I do touch — the tickets, the paperwork and the Linux side of the machine. Where something is a genuine hardware-handling skill, I have said so and pointed at the people who have it, rather than pretending otherwise.

Next up: Part 1 — anatomy of a rack, which starts with the fact that a "42U cabinet" does not give you 42U of usable space.