Back-End Principles: HTTP Deep Dive

HTTP (HyperText Transfer Protocol) is the foundation of data communication on the web. It defines how clients (usually browsers) and servers communicate through requests and responses.

---

What is HTTP?

HTTP (HyperText Transfer Protocol) is the foundation of data communication on the web. It defines how clients (usually browsers) and servers communicate through requests and responses.

Key Characteristics

Stateless:

Each request is independent — the server doesn’t remember previous requests. To manage state across multiple requests, we use:

• Sessions
• Cookies
• Tokens (JWT, OAuth, etc.)

Client-Server Model:

The client sends a request; the server processes it and sends back a response.

---

HTTP Request Components

An HTTP request consists of:

1. Start Line – Defines method, URL, and protocol version.
2. Headers – Provide metadata about the request.
3. Body (optional) – Contains data sent to the server (for POST, PUT, etc.).

Example HTTP Requests

GET request (no body):

GET /api/users?page=2 HTTP/1.1
Host: api.example.com
Accept: application/json
User-Agent: MyClient/1.0

POST request (JSON body):

POST /api/users HTTP/1.1
Host: api.example.com
Content-Type: application/json; charset=utf-8
Authorization: Bearer <token>
Content-Length: 38

{"email":"alice@example.com","role":"admin"}

Common HTTP Headers

Header	Description	Example
User-Agent	Identifies the client making the request.	Mozilla/5.0 (Windows NT 10.0; Win64; x64)
Authorization	Carries credentials like tokens or API keys.	Authorization: Bearer <token>
Cookie	Stores session data on the client.	Cookie: session_id=1234
Accept	Defines acceptable response formats.	Accept: application/json
Content-Type	Describes the body format.	Content-Type: application/json
Cache-Control	Manages caching policies.	Cache-Control: no-cache
Connection	Controls connection behavior.	Connection: keep-alive

Response Headers (Security & Representation)

Header	Description	Example
Content-Type	Format of returned data.	Content-Type: application/json
Content-Length	Size of the response body.	Content-Length: 348
ETag	Identifier for caching validation.	ETag: "abc123"
Strict-Transport-Security	Forces HTTPS.	Strict-Transport-Security: max-age=31536000
X-Content-Type-Options	Prevents MIME sniffing.	X-Content-Type-Options: nosniff

---

HTTP Response Components

An HTTP response consists of:

1. Status Line – Protocol version, status code, and reason phrase (e.g., HTTP/1.1 200 OK).
2. Headers – Metadata about the response (content type, caching, cookies, etc.).
3. Body (optional) – The returned content (JSON, HTML, file bytes, etc.).

Example HTTP Response

HTTP/1.1 200 OK
Date: Sat, 01 Nov 2025 12:00:00 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 42
ETag: "abc123"
Set-Cookie: session_id=98a2f0c1; HttpOnly; Secure; Path=/; SameSite=Lax

{"id":1,"name":"Alice"}

Notes and Special Cases

• Responses to HEAD and status codes 204 No Content, 304 Not Modified, and all 1xx informational responses MUST NOT include a message body.
• Redirects (3xx) often include a Location header indicating the next URL.
• File downloads typically set Content-Disposition to suggest a filename.
• Binary bodies should set the appropriate Content-Type (e.g., application/pdf, image/png).

HTTP Extensibility

HTTP allows adding custom headers and new methods without breaking compatibility.

Example:

X-Request-ID: 98a2f0c1

Used for tracing requests across services in distributed systems.

---

Remote Control

HTTP can be used to control remote systems via APIs — not just web pages.

Example: Sending a POST request to start a server task:

POST /api/restart

---

HTTP Methods

Method	Description	Example
GET	Retrieve data.	/users → Get all users
POST	Create new data.	/users → Add a new user
PUT	Replace an existing resource completely.	/users/1 → Update all user info
PATCH	Update part of a resource.	/users/1 → Update only email
DELETE	Remove a resource.	/users/1 → Delete user
OPTIONS	Ask server which methods are supported.	/users → Returns allowed methods

---

Idempotent vs Non-Idempotent Methods

Idempotency means that performing the same action repeatedly has the same effect as doing it once.

• Safe methods: GET, HEAD, and OPTIONS are intended to be read-only (no side effects) though servers may still log or count them.
• Idempotent methods: GET, PUT, DELETE, HEAD, OPTIONS should be implemented so that retries don’t change state beyond the first success.
• Non-idempotent: POST (and often PATCH) may create new resources or apply additional changes on each call.

Type	Methods	Description	Example
Safe & Idempotent	GET, HEAD, OPTIONS	No resource modification expected.	GET /users/1 for the same representation.
Idempotent (not safe)	PUT, DELETE	Same call repeated leads to the same final state.	PUT /users/1 with the same body overwrites to the same value; repeated DELETE /users/1 still results in no user.
Non-Idempotent	POST (often PATCH)	Repeating may create or mutate multiple times.	Two POST /orders calls create two orders.

Practical guidance:

• Implement retries for idempotent operations transparently at clients/gateways.
• For POST endpoints that must be retryable (e.g., payments), use an Idempotency-Key header and deduplicate on the server.
• Document whether PATCH is idempotent in your API; behavior varies by design.

---

OPTIONS Method & CORS

OPTIONS serves two common purposes:

1. Discover server capabilities for a resource (allowed methods, headers).
2. Handle CORS preflight requests from browsers.

CORS Preflight Request

Browser sends (to the target origin):

OPTIONS /api/resource HTTP/1.1
Origin: https://app.example.com
Access-Control-Request-Method: POST
Access-Control-Request-Headers: Authorization, Content-Type

Server responds with what is allowed:

HTTP/1.1 204 No Content
Access-Control-Allow-Origin: https://app.example.com
Access-Control-Allow-Methods: GET, POST
Access-Control-Allow-Headers: Authorization, Content-Type
Access-Control-Max-Age: 3600

If the preflight allows it, the browser proceeds with the actual request.

---

CORS (Cross-Origin Resource Sharing)

CORS allows servers to specify which origins can access their resources from a different domain.

Example Response:

Access-Control-Allow-Origin: https://example.com
Access-Control-Allow-Methods: GET, POST, PUT
Access-Control-Allow-Headers: Content-Type, Authorization

Without correct CORS headers, browsers will block requests from other domains.

Additional notes:

• Simple requests (GET, HEAD, POST with Content-Type of application/x-www-form-urlencoded, multipart/form-data, or text/plain and limited headers) skip preflight.
• To send cookies/credentials, the server must set Access-Control-Allow-Credentials: true and the response must use a specific origin (no *). Client must use fetch(..., { credentials: "include" }).
• Use Vary: Origin when responses differ per origin to ensure proper caching behavior at CDNs/proxies.
• Access-Control-Max-Age allows caching of preflight results to reduce latency.

---

HTTP Status Codes

Category	Meaning	Example Codes	Example
1xx	Informational	100 Continue, 101 Switching Protocols	Connection setup
2xx	Success	200 OK, 201 Created, 204 No Content	Successful request
3xx	Redirection	301 Moved Permanently, 302 Found, 304 Not Modified	Resource moved or cached
4xx	Client Errors	400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found	User input or access issues
5xx	Server Errors	500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable	Server or proxy issues

---

HTTP Caching

Caching improves performance by storing responses for reuse.

Headers Used:

• Cache-Control
• ETag
• Last-Modified
• Expires

Example:

Cache-Control: max-age=3600
ETag: "v1.2"

This tells browsers to reuse cached data for 1 hour unless changed.

---

Content Negotiation

Allows clients and servers to agree on data format or language.

Example:

Accept: application/json
Accept-Language: en-US

The server may respond with:

Content-Type: application/json
Content-Language: en-US

Notes:

• Clients can provide preferences using quality weights (q-values): Accept: application/json; q=1.0, text/html; q=0.8.
• Content negotiation can be server-driven (server picks best match from headers) or agent-driven (client chooses among variants, e.g., via links).
• Other negotiable dimensions include Accept-Charset and Accept-Encoding.
• When serving different representations, include Vary (e.g., Vary: Accept, Accept-Language) to keep intermediaries caching correctly.

---

HTTP Compression

Reduces response size using algorithms like gzip or br.

Example:

Accept-Encoding: gzip, deflate, br
Content-Encoding: gzip

---

Persistent Connections & Keep-Alive

HTTP/1.1 introduced persistent connections, allowing multiple requests over a single TCP connection.

Example:

Connection: keep-alive
Keep-Alive: timeout=5, max=100

More details:

• Persistent connections avoid the cost of repeated TCP/TLS handshakes and improve latency via connection reuse (a.k.a. connection pooling).
• Keep-Alive is advisory; servers may still close idle connections at any time. Clients should retry transparently.
• HTTP/1.1 supports pipelining in theory but it is rarely used due to head-of-line blocking.
• With HTTP/2 and HTTP/3, a single connection can multiplex many concurrent streams; explicit Keep-Alive headers are unnecessary, but connection reuse remains critical for performance.

---

Multipart Data & Chunked Transfer

Multipart Form Data: Used for file uploads.

Content-Type: multipart/form-data; boundary=---12345

Chunked Transfer Encoding: Sends data in chunks when the total size is unknown.

Transfer-Encoding: chunked

Examples:

Multipart body (two fields + one file):

POST /upload HTTP/1.1
Host: files.example.com
Content-Type: multipart/form-data; boundary=---12345

---12345
Content-Disposition: form-data; name="description"

Holiday photo
---12345
Content-Disposition: form-data; name="photo"; filename="beach.png"
Content-Type: image/png

<binary bytes>
---12345--

Chunked body (sizes are hex):

HTTP/1.1 200 OK
Transfer-Encoding: chunked

4\r\n
Wiki\r\n
5\r\n
pedia\r\n
0\r\n
\r\n

With chunked encoding, optional trailers can follow the final 0 chunk if Trailer/TE: trailers are used.

---

SSL, TLS, and HTTPS

Protocol	Description
SSL (Secure Sockets Layer)	Outdated encryption protocol.
TLS (Transport Layer Security)	Successor to SSL, provides encryption & integrity.
HTTPS	HTTP over TLS — secures communication between client and server.

Example:

https://example.com

All data exchanged is encrypted and protected from eavesdropping.

How HTTPS works (simplified):

1. Client initiates a TLS handshake (ClientHello with supported versions/ciphers and SNI for the hostname).
2. Server responds with its certificate chain and parameters.
3. Client validates the certificate chain against trusted roots and hostname; if valid, both sides derive shared keys.
4. An encrypted channel is established and HTTP messages flow over it.

Operational notes:

• Use strong modern TLS versions (TLS 1.2+) and disable legacy ciphers.
• Configure HSTS to force HTTPS in browsers: Strict-Transport-Security: max-age=31536000; includeSubDomains.
• Certificates can be automated via ACME (e.g., Let’s Encrypt). Rotate before expiry.

---

Summary

Understanding HTTP is essential for back-end engineers. From request/response cycles to security headers, caching strategies, and protocol evolution, HTTP remains the backbone of web communication. Master these concepts to build robust, secure, and performant web applications.