Architecting Large Web Applications: A Day One Guide
A practitioner’s guide to building scalable, maintainable applications before the first line of production code.
Introduction: Why Architecture Matters
Every developer has been there. You join a growing startup, open the codebase, and immediately feel a sinking sensation. Files are scattered everywhere. Controllers contain entire business flows. The database schema is held together by duct tape. Changing one feature breaks three others. And the scariest question of all is: how did this happen?
The answer is almost always the same: nobody planned for scale. The original developers moved fast, and that’s understandable: in the early days of a product, speed matters. But speed without structure is a debt that compounds rapidly. Every shortcut taken in week one becomes a headache in month six and a full-blown crisis by year two.
Consider the experience of companies like Shopify, Slack, or Airbnb. All three started as relatively simple applications. All three hit severe architectural growing pains. Shopify famously had to re-architect significant parts of its monolith under the pressure of Black Friday traffic spikes. Slack spent years untangling a tightly coupled codebase as it scaled from thousands to millions of daily users. These are companies with world-class engineering teams: and they still paid the price of early architectural shortcuts.
The good news is that you don’t have to be building the next Shopify to benefit from solid architectural thinking. Good architecture isn’t about over-engineering. It’s about making deliberate decisions early that cost you very little now but save you enormously later.
The cost of bad architecture decisions compounds like interest. A structural choice made in sprint one will be touched by every developer, in every feature, for the entire life of your product. Getting it right: or at least reasonable: from the start is one of the highest-ROI investments a team can make.
This guide is for developers at every level: whether you’re a junior engineer building your first serious application, a technical co-founder bootstrapping a product, or a senior engineer joining a new team and trying to establish good practices from day one. We’ll cover everything: folder structure, service layers, dependency injection, design patterns, scalability planning, and the common traps that even experienced engineers fall into.
Section 1: What Does “Architecting from Day One” Mean?
When people hear “architect from day one,” they sometimes assume it means spending weeks drawing diagrams before writing a line of code. That’s not it. Architecting from day one means making intentional structural decisions upfront so that your codebase can grow without collapsing under its own weight.
Building for Growth
Designing for growth doesn’t mean assuming your app will have a million users next month. It means writing code in a way that won’t actively prevent you from reaching that milestone. The key is modularity: if your features are self-contained and your layers are clearly separated, adding new features, fixing bugs, or onboarding new developers becomes dramatically easier.
Avoiding Premature Optimization
There’s a balance to strike. Donald Knuth’s famous quote: “premature optimization is the root of all evil”: is often invoked to justify messy code. But there’s a difference between premature optimization (caching every database query before you know you have a performance problem) and foundational architecture (putting your database queries behind a repository layer so you can optimize them later without touching your business logic).
Do the second. Avoid the first.
Balancing Simplicity and Scalability
The best architectures are deceptively simple. They’re not clever: they’re clear. A well-architected application should be readable by a new developer on their first day. Clear folder structures, predictable naming conventions, and consistent patterns reduce cognitive load and dramatically speed up development.
Rule of thumb: If a new developer can open your project and understand where to add a new feature within 15 minutes, your architecture is doing its job.
Architectural Thinking as a Skill
Architecture isn’t a one-time event. It’s an ongoing mindset. The best engineers are always thinking about the structure of what they’re building, not just the immediate task. They ask: “Does this decision make the codebase easier or harder to change?” If the answer is harder, they look for a better approach: or at least document the tradeoff.
Section 2: Core Principles of Large Application Architecture
Great software architecture isn’t invented from scratch for each project. It’s built on a set of well-proven principles. These principles have been refined over decades of real-world software engineering. Let’s walk through the most important ones.
Separation of Concerns (SoC)
Each part of your application should have one clear responsibility, and that responsibility shouldn’t bleed into other parts. Your routing layer handles HTTP requests. Your service layer handles business logic. Your database layer handles data persistence. When you mix these, you create code that’s nearly impossible to test or reason about.
Single Responsibility Principle (SRP)
Every class, function, and module should do exactly one thing and do it well. A UserService manages user-related business logic. A UserRepository manages database queries for users. A UserController handles HTTP requests related to users. None of these should reach into each other’s domain.
Modularity
Large applications should be composed of self-contained modules. Each module: think “users,” “orders,” “payments”: should own its own controllers, services, repositories, and models. This makes it possible for different teams to work independently without stepping on each other’s toes.
Loose Coupling
Components should know as little about each other as possible. If your OrderService directly instantiates a PaymentService inside its methods, they are tightly coupled. A change to PaymentService can break OrderService in unexpected ways. Loose coupling means components interact through abstractions (interfaces, contracts) rather than concrete implementations.
High Cohesion
Related code should live together. Everything related to user authentication: the models, the services, the validation logic, the routes: should be in one place, not scattered across the codebase. High cohesion makes navigation intuitive and changes safer.
Scalability
The system should be designed to handle growth: in users, data, traffic, and team size. This doesn’t mean implementing Kubernetes on day one. It means writing stateless services, using environment-based configuration, and not baking assumptions about scale into the core logic.
Maintainability
Code is read far more often than it’s written. Clear naming, consistent patterns, and well-organized structure make maintenance faster and safer. A maintainable codebase is one where you can confidently make changes without fear of unexpected breakage.
Testability
If your code is hard to test, that’s usually a sign that it’s hard to understand and hard to change. Testability is a proxy for design quality. Applications designed with proper separation of concerns and dependency injection are naturally easy to unit test, integration test, and end-to-end test.
Section 3: Designing the Project Folder Structure
Folder structure is one of the first signals of an application’s architectural quality. It’s also one of the most underestimated. How you organize your code shapes how developers navigate, reason about, and extend your application.
The Classic Mistake: Organizing by Type
Almost every tutorial starts with something like this:
project/
controllers/
userController.js
orderController.js
productController.js
models/
User.js
Order.js
Product.js
services/
userService.js
orderService.js
utils/
helpers.js
This feels organized: and for a tiny app, it might be fine. But as the application grows, this structure becomes painful. To implement a new feature for orders, you have to touch files spread across four different folders. There’s no clear boundary between features. Every developer is working in the same controllers/ folder, leading to constant merge conflicts. Onboarding a new developer requires understanding the entire codebase at once because nothing is self-contained.
Organizing by type is fine for tutorials and tiny apps. It becomes a maintenance nightmare at scale. When a single feature’s code is scattered across five folders, you lose the ability to reason about that feature in isolation.
The Scalable Approach: Feature-Based (Module-Based) Structure
The industry-proven solution is to organize by feature (or module), not by type. Each feature owns everything it needs:
src/
├── modules/
│ ├── users/
│ │ ├── user.controller.ts
│ │ ├── user.service.ts
│ │ ├── user.repository.ts
│ │ ├── user.model.ts
│ │ ├── user.dto.ts
│ │ ├── user.routes.ts
│ │ └── user.spec.ts
│ ├── products/
│ │ ├── product.controller.ts
│ │ ├── product.service.ts
│ │ ├── product.repository.ts
│ │ ├── product.model.ts
│ │ └── product.routes.ts
│ ├── orders/
│ └── payments/
├── shared/
│ ├── middleware/
│ ├── utils/
│ ├── validators/
│ └── types/
├── infrastructure/
│ ├── database/
│ ├── cache/
│ ├── mailer/
│ └── queue/
├── config/
│ ├── app.config.ts
│ ├── database.config.ts
│ └── env.ts
└── tests/
├── integration/
└── e2e/
What Each Folder Does
- modules/: Each subdirectory is a self-contained feature. Everything related to “users” lives in
modules/users/. New developers can understand and work on a feature without touching anything else. - shared/: Code that is genuinely cross-cutting: middleware that applies to all routes, validation utilities used by multiple modules, shared TypeScript types and interfaces.
- infrastructure/: Adapters to external systems: the database connection, cache (Redis), email service, message queue (RabbitMQ, SQS). This layer exists so that swapping a database or queue system only requires changing one folder.
- config/: All application configuration in one place. Reads from environment variables, never hardcodes values. Provides typed configuration objects to the rest of the app.
- tests/: Integration and end-to-end tests that span multiple modules. Unit tests live alongside their respective module files.
Section 4: Understanding Service Layer Architecture
One of the most important structural decisions you’ll make is how to organize the layers within each module. The most proven approach in enterprise web development is a layered architecture: Controller → Service → Repository → Database.
The Request Flow
Each layer has a single, well-defined job:
- Controller: Receives HTTP requests, validates input, calls the appropriate service method, and returns HTTP responses. Contains no business logic.
- Service: Contains all business logic. Orchestrates repositories and other services. Knows nothing about HTTP.
- Repository: Abstracts all database access. Knows SQL (or ORM queries). Knows nothing about business rules.
- Database: The actual data store (PostgreSQL, MongoDB, etc.).
Bad Implementation
This is what you’ll find in many tutorial-style codebases:
// ❌ Fat Controller: everything mixed together
app.post('/orders', async (req, res) => {
const { userId, productId, quantity } = req.body;
// Business logic inside controller
const product = await db.query('SELECT * FROM products WHERE id = ?', [productId]);
if (product.stock < quantity) {
return res.status(400).json({ error: 'Insufficient stock' });
}
const total = product.price * quantity;
// Directly calling payment API from controller
const payment = await stripe.charges.create({ amount: total, currency: 'usd' });
await db.query('INSERT INTO orders (user_id, product_id, total) VALUES (?, ?, ?)',
[userId, productId, total]);
res.json({ success: true, orderId: payment.id });
});
This controller is doing everything: validating business rules, querying the database, calling an external API. It's nearly impossible to unit test and will become a maintenance nightmare.
Better Implementation: Layered Architecture
// order.controller.ts: Only handles HTTP concerns
@Controller('/orders')
export class OrderController {
constructor(private readonly orderService: OrderService) {}
@Post('/')
async createOrder(@Body() dto: CreateOrderDto, @Req() req: Request) {
const order = await this.orderService.createOrder(req.user.id, dto);
return { success: true, order };
}
}
// order.service.ts: Contains business logic
@Injectable()
export class OrderService {
constructor(
private readonly orderRepository: OrderRepository,
private readonly productRepository: ProductRepository,
private readonly paymentService: PaymentService,
) {}
async createOrder(userId: string, dto: CreateOrderDto): Promise {
const product = await this.productRepository.findById(dto.productId);
if (!product) throw new NotFoundException('Product not found');
if (product.stock < dto.quantity) {
throw new BadRequestException('Insufficient stock');
}
const total = product.price * dto.quantity;
const payment = await this.paymentService.charge(userId, total);
return this.orderRepository.create({
userId, productId: dto.productId, total, paymentId: payment.id,
});
}
}
// order.repository.ts: Only database access
@Injectable()
export class OrderRepository {
constructor(private readonly db: DatabaseService) {}
async create(data: CreateOrderData): Promise {
return this.db.orders.create({ data });
}
async findByUserId(userId: string): Promise {
return this.db.orders.findMany({ where: { userId } });
}
}
Now each layer has exactly one job. The controller is trivially simple. The service contains readable business logic. The repository is just database queries. Each layer is independently testable.
Section 5: Dependency Injection Explained
Dependency Injection (DI) is one of those concepts that sounds intimidating when you first encounter it, but once you understand it, you'll wonder how you ever wrote code without it.
What is Dependency Injection?
At its core, DI is simple: instead of a class creating its own dependencies, dependencies are provided to the class from the outside. The class declares what it needs; something else (a DI container, or your own wiring code) provides it.
Without Dependency Injection
// ❌ Without DI: tightly coupled
class OrderService {
private paymentService = new PaymentService();
private orderRepository = new OrderRepository();
async createOrder(data: CreateOrderData) {
// Uses the concrete PaymentService directly
const payment = await this.paymentService.charge(data.total);
return this.orderRepository.create({ ...data, paymentId: payment.id });
}
}
Problems with this approach:
- You can't unit test
OrderServicewithout also runningPaymentServiceandOrderRepository. - If
PaymentServicerequires a network connection (to Stripe, for example), every test requires a live network. - Swapping
PaymentServicefor a different implementation means modifyingOrderService.
With Dependency Injection
// ✅ With DI: loosely coupled
class OrderService {
constructor(
private readonly paymentService: IPaymentService,
private readonly orderRepository: IOrderRepository,
) {}
async createOrder(data: CreateOrderData) {
const payment = await this.paymentService.charge(data.total);
return this.orderRepository.create({ ...data, paymentId: payment.id });
}
}
// In tests, inject mocks:
const mockPaymentService: IPaymentService = {
charge: jest.fn().mockResolvedValue({ id: 'ch_test_123' }),
};
const mockOrderRepository: IOrderRepository = {
create: jest.fn().mockResolvedValue({ id: 'ord_1', ...testData }),
};
const service = new OrderService(mockPaymentService, mockOrderRepository);
// Now you can test business logic in complete isolation
Enterprise DI with a Container
In larger applications, manually wiring dependencies becomes verbose. DI containers (like NestJS's built-in container, InversifyJS, or tsyringe) automate this:
// NestJS DI container example
@Module({
providers: [OrderService, OrderRepository, PaymentService],
controllers: [OrderController],
})
export class OrderModule {}
// NestJS automatically injects the right instances everywhere
DI is the foundation of testable, maintainable code. Any application that expects to grow beyond a few thousand lines of code should adopt a DI pattern early. The overhead is minimal; the benefits are enormous.
Section 6: Design Patterns Every Large Application Should Use
Design patterns are proven, reusable solutions to commonly occurring problems in software design. They're not code you copy-paste: they're blueprints for how to structure code to solve specific categories of problems. Here are the most valuable patterns for large web applications.
Repository Pattern
Abstracts the data layer from business logic. Your services talk to a repository interface, not directly to the database. This means you can swap PostgreSQL for MongoDB without touching a single service.
interface IUserRepository {
findById(id: string): Promise<User | null>;
create(data: CreateUserDto): Promise<User>;
}
Decouples business logic from databaseAdds abstraction overhead for tiny apps
Factory Pattern
Centralizes object creation. Instead of new scattered everywhere, a factory decides which concrete implementation to instantiate based on context.
class PaymentFactory {
static create(type: 'stripe' | 'paypal') {
if (type === 'stripe') return new StripeService();
return new PayPalService();
}
}
Easy to add new typesFactory can become a god class if overused
Singleton Pattern
Ensures a class has only one instance throughout the application. Classic use cases: database connection pools, configuration objects, logging instances.
class DatabaseConnection {
private static instance: DatabaseConnection;
static getInstance() {
if (!this.instance) {
this.instance = new DatabaseConnection();
}
return this.instance;
}
}
Prevents duplicate connectionsHard to test; avoid for business logic
Strategy Pattern
Defines a family of algorithms (strategies) and makes them interchangeable. Ideal for payment processing, shipping rate calculation, or discount rules that vary by context.
interface ShippingStrategy {
calculate(order: Order): number;
}
class StandardShipping implements ShippingStrategy {
calculate(order: Order) { return 5.99; }
}
class ExpressShipping implements ShippingStrategy {
calculate(order: Order) { return 19.99; }
}
Open/Closed Principle: add strategies without changing existing code
Observer Pattern
Objects subscribe to events emitted by other objects. Core to event-driven architecture. When an order is placed, observers (email service, inventory service, analytics) are notified automatically.
class EventEmitter {
private listeners: Map<string, Function[]> = new Map();
on(event: string, listener: Function) {
const list = this.listeners.get(event) || [];
this.listeners.set(event, [...list, listener]);
}
emit(event: string, data: any) {
(this.listeners.get(event) || []).forEach(fn => fn(data));
}
}
Loose coupling between producers and consumers
Adapter Pattern
Wraps an incompatible interface in a compatible one. When you integrate a third-party service, wrap it in an adapter so the rest of your app never directly depends on the vendor's API shape.
class StripeAdapter implements IPaymentGateway {
async charge(amount: number, currency: string) {
// Translates your interface to Stripe's API
return stripe.paymentIntents.create({ amount, currency });
}
}
Protects your codebase from vendor changes
Command Pattern
Encapsulates a request as an object. Useful for queueable tasks, undo operations, and audit logs. Each command is a self-contained unit of work.
interface Command {
execute(): Promise<void>;
}
class SendOrderConfirmationCommand implements Command {
constructor(private readonly order: Order) {}
async execute() {
await emailService.send(this.order.userEmail, 'Order confirmed!');
}
}
Decouples sender from receiver; supports queuing and retry
Section 7: Future Scalability Considerations
You don't need to implement all of these on day one. But you should design your system so that adding these doesn't require a complete rewrite. Think of it as leaving the right doors open.
Stateless Services
Your application servers should not store any session data in memory. All state: user sessions, shopping carts, temporary data: should be stored in an external store (Redis, for example). Stateless servers can be horizontally scaled by simply adding more instances. If your servers store state in memory, scaling becomes a nightmare.
Horizontal vs Vertical Scaling
| Scaling Type | What It Means | When to Use | Limitations |
|---|---|---|---|
| Vertical | Bigger, faster server | Quick wins, small scale | Physical hardware limits; expensive; single point of failure |
| Horizontal | More server instances | Production scale | Requires stateless services; adds complexity |
Caching
Caching is one of the most impactful performance optimizations available. Implement it at multiple levels:
- Application-level cache (Redis): Cache expensive database queries, computed results, and API responses.
- HTTP cache headers: Tell browsers and CDNs to cache static responses.
- CDN caching: Serve static assets and pre-rendered pages from edge locations close to your users.
Database Replication and Sharding
Read replicas allow you to distribute read traffic across multiple database instances, massively increasing read throughput. Sharding splits data across multiple database servers by some key (like user region or user ID range) and is used when a single database server can no longer handle your data volume.
Load Balancing
A load balancer sits in front of your application servers and distributes incoming requests across instances. This enables horizontal scaling and provides failover: if one instance goes down, traffic is automatically rerouted to healthy instances.
Message Queues and Event-Driven Architecture
For operations that don't need to happen synchronously: sending emails, generating reports, processing images: use a message queue (RabbitMQ, SQS, Kafka). The API responds immediately; the heavy work happens asynchronously in the background. This dramatically improves responsiveness and decouples services.
// Instead of doing this synchronously in the request handler:
await emailService.sendOrderConfirmation(order); // ❌ Blocks the request
// Do this: publish an event and let a worker handle it:
await queue.publish('order.created', { orderId: order.id }); // ✅ Non-blocking
// A separate worker process consumes this event and sends the email
Section 8: Case Study: Building an E-Commerce Platform
Let's make this concrete. Suppose you're building an e-commerce platform. It needs to handle product browsing, user accounts, shopping carts, orders, payments, reviews, and notifications. Here's how you'd architect it from day one.
Module Breakdown
src/modules/
├── auth/ : Authentication, JWT, sessions
├── users/ : User profiles, preferences
├── products/ : Catalog, search, categories
├── inventory/ : Stock levels, warehouse locations
├── cart/ : Cart management (Redis-backed)
├── orders/ : Order lifecycle, status management
├── payments/ : Payment processing (Stripe adapter)
├── reviews/ : Product reviews, ratings
└── notifications/ : Email, SMS, push notification orchestration
Domain Models
// User
interface User {
id: string;
email: string;
passwordHash: string;
role: 'customer' | 'admin';
createdAt: Date;
}
// Product
interface Product {
id: string;
name: string;
description: string;
price: number; // stored in cents to avoid floating point issues
categoryId: string;
stock: number;
images: string[];
isActive: boolean;
}
// Order
interface Order {
id: string;
userId: string;
items: OrderItem[];
total: number;
status: 'pending' | 'confirmed' | 'shipped' | 'delivered' | 'cancelled';
paymentId: string;
shippingAddress: Address;
createdAt: Date;
}
API Structure
POST /api/v1/auth/register
POST /api/v1/auth/login
POST /api/v1/auth/refresh
GET /api/v1/products?page=1&limit=20&category=electronics
GET /api/v1/products/:id
POST /api/v1/products (admin only)
GET /api/v1/cart
POST /api/v1/cart/items
DELETE /api/v1/cart/items/:productId
POST /api/v1/orders (creates order from cart)
GET /api/v1/orders/:id
GET /api/v1/users/me/orders
POST /api/v1/payments/intent (creates Stripe PaymentIntent)
POST /api/v1/payments/webhook (Stripe webhook handler)
Future Scaling Strategy
Starting with this modular monolith, scaling milestones would look like:
- 0-10k users: Single server, PostgreSQL, Redis for cart/sessions, CDN for assets.
- 10k-100k users: Add read replicas, introduce a job queue for emails/notifications, add horizontal scaling behind a load balancer.
- 100k+ users: Extract notifications and search into separate services. Add Elasticsearch for product search. Consider extracting the most independently scaling modules into microservices.
The beauty of a well-structured modular monolith is that each module is already a microservice waiting to happen. When you're ready to extract one, the boundaries are already defined.
Section 9: Common Architecture Mistakes Developers Make
| Mistake | Symptom | Fix |
|---|---|---|
| God Classes | One class that does everything (3,000-line UserService) |
Apply SRP; break into focused classes |
| Fat Controllers | Business logic, DB queries, and HTTP all in one place | Move logic to services; keep controllers thin |
| Tight Coupling | Changing one class breaks several others | Program to interfaces; use DI |
| No Abstraction Layers | Switching databases requires rewriting the whole app | Use repository pattern; abstract external dependencies |
| Business Logic in Database Layer | Complex stored procedures and triggers | Business rules belong in the service layer |
| Premature Microservices | 5 developers managing 20 services with Kubernetes | Start with a modular monolith; extract services when you have clear need |
| Poor Naming Conventions | doStuff(), helper2.js, DataManager |
Names should reveal intent; use consistent conventions |
| No Tests | Every deploy is a prayer | Test from the start; well-architected code is naturally testable |
Premature microservices is one of the most common and costly mistakes we see in startups. Microservices add enormous operational complexity: you need service discovery, inter-service authentication, distributed tracing, and a deployment pipeline for each service. A team of five doesn't need this. Build a solid monolith first.
Section 10: When Should You Move to Microservices?
Microservices are not an architecture for startups or small teams. They are an organizational scaling solution as much as a technical one. Before making the switch, you need to have very specific, validated reasons.
| Monolith | Microservices | |
|---|---|---|
| Team Size | 1-20 developers | Large teams, multiple squads |
| Deployment | Simple, one unit | Complex; each service independent |
| Development Speed | Fast initially | Slower setup; faster at scale |
| Operational Complexity | Low | High (orchestration, monitoring) |
| Scaling Granularity | Scale the whole app | Scale individual services |
| Testing | Straightforward | Requires contract testing, integration suites |
| Best For | Early-stage, product-market fit | Post-scale, multiple independent domains |
The Decision Framework
Consider extracting a service when all of the following are true:
- You have a clearly bounded domain that changes independently and frequently.
- The domain needs to scale differently from the rest of the application.
- You have a team (or soon will have a team) dedicated to owning that domain.
- The interface between this domain and the rest of your system is stable and well-defined.
If any of those conditions is missing, stay with the monolith and invest in making it cleaner instead.
Section 11: Architecture Checklist for New Projects
Before writing production code on a new project, work through this checklist:
Structure & Organization
- Use a feature-based (module-based) folder structure, not type-based
- Each module is self-contained with its own controller, service, repository, model, and tests
- Shared utilities and cross-cutting concerns live in a
/shareddirectory - Infrastructure adapters (database, cache, email) live in an
/infrastructurelayer - All configuration is environment-variable driven; no hardcoded values
Design Principles
- Controllers are thin: they only handle HTTP concerns
- All business logic lives in service classes
- All database access goes through repository classes
- Dependencies are injected, not instantiated inside classes
- External services are wrapped in adapter classes
- Every class and function has a single, clear responsibility
Testing
- Unit tests cover all service-layer business logic
- Integration tests cover critical API endpoints
- Tests can run without a live database (using mocks/stubs)
- CI pipeline runs tests on every pull request
Scalability
- Application servers are stateless (no in-memory session storage)
- Session/cart data stored in Redis or equivalent external store
- Long-running tasks (emails, notifications) use an async job queue
- Database queries go through the repository pattern for easy swapping
- API versioned from the start (
/api/v1/)
Security & Reliability
- All secrets in environment variables, never in code or version control
- Input validation on all API endpoints (DTOs with class-validator)
- Structured logging with a correlation/request ID for tracing
- Health check endpoint (
/health) for load balancers and monitoring - Rate limiting on public-facing endpoints
Frequently Asked Questions
const service = new PaymentService() inside your class, the PaymentService is passed in through the constructor. This makes code dramatically more testable (you can pass in mocks), more flexible (you can swap implementations), and easier to understand.src/modules/users/ contains the user controller, service, repository, model, routes, and tests. This makes features self-contained, reduces merge conflicts, and makes onboarding much faster.userRepository.findById(id)). The repository implementation handles the actual database query. This means you can swap databases, use different ORMs, or mock the database entirely in tests: without touching any business logic.Conclusion
Architecture isn't about being clever. It's about being clear. The best-architected systems are the ones where every developer: from the intern on their first day to the principal engineer who's been there for years: can open the codebase and immediately understand where to find things and where to add new things.
The principles we've covered in this guide aren't new. They've been refined by thousands of engineers across thousands of projects over decades. They work because they align with how software actually grows: organically, unpredictably, and often much faster than anyone planned for.
Start with a clean folder structure organized by feature. Separate your controllers, services, and repositories into distinct layers. Use dependency injection so your code is testable and your dependencies are swappable. Learn the core design patterns and reach for them when they naturally fit. Design your services to be stateless so scaling is a matter of adding instances rather than rewriting code.
The investment you make in architecture on day one is the most leveraged investment you'll make for the entire life of your application. Every hour spent on good structure now saves dozens of hours in debugging, refactoring, and explaining to confused teammates down the road.
So before you type npm init on your next project, take a breath, sketch out your modules, and think about the structure you're building toward. Your future self: and your teammates: will thank you.
Key Takeaways
Organize by feature, not by type. Self-contained modules scale with your team.
Separate controllers, services, and repositories. Each layer has exactly one job.
Inject dependencies; don't instantiate them. Testability reveals design quality.
Learn design patterns. They're proven solutions to recurring architectural problems.
Design for scale from the start. Stateless services, queues, and abstracted infrastructure make growth manageable.
Start monolith, think modules. Microservices are an organizational solution, not a technical starting point.
Bonus Section A: Designing a Scalable API from Day One
Your API is the public face of your application's architecture. A well-designed API is predictable, versioned, and expressive. A poorly designed one becomes a source of confusion for every client: frontend teams, mobile developers, and third-party integrators: for the entire life of the product. Getting it right early costs almost nothing. Fixing it later (while maintaining backward compatibility) is one of the most painful migrations a team can go through.
API Versioning
Version your API from the very first endpoint. Even if you never change a thing in v1, having the version in the URL gives you an escape hatch when you inevitably need to make a breaking change.
// ✅ Always version from the start
GET /api/v1/users
POST /api/v1/orders
// When breaking changes are needed, v2 routes coexist with v1
GET /api/v2/users // new shape, new behaviour
GET /api/v1/users // still works for existing clients
A popular alternative to URL versioning is header-based versioning (Accept: application/vnd.myapp.v2+json), but URL versioning is simpler, more cacheable, and easier to document for most teams.
RESTful Resource Naming
Resources are nouns, not verbs. Actions are expressed through HTTP methods. This is one of the most commonly violated REST principles in real-world codebases:
| Wrong (verb-based) | Correct (noun + HTTP method) |
|---|---|
POST /createUser | POST /api/v1/users |
GET /getUserById?id=5 | GET /api/v1/users/5 |
POST /deleteOrder | DELETE /api/v1/orders/12 |
GET /getProductReviews | GET /api/v1/products/7/reviews |
POST /cancelSubscription | PATCH /api/v1/subscriptions/3 with { "status": "cancelled" } |
Consistent Response Envelopes
Define a consistent response structure for every API response: success and error. This makes client-side handling predictable and enables middleware-level logging and error reporting.
// ✅ Consistent success response envelope
{
"success": true,
"data": {
"id": "usr_abc123",
"email": "alice@example.com",
"createdAt": "2026-01-15T10:30:00Z"
},
"meta": {
"page": 1,
"limit": 20,
"total": 847
}
}
// ✅ Consistent error response envelope
{
"success": false,
"error": {
"code": "VALIDATION_ERROR",
"message": "Email address is already in use.",
"details": [
{ "field": "email", "message": "Must be unique" }
]
},
"requestId": "req_xk9f2a" // trace ID for debugging
}
The requestId (or trace ID) field is especially valuable in production: when a user reports an error, you can search your logs for that ID and get the complete request context instantly.
HTTP Status Codes: Use Them Correctly
Many APIs return 200 OK for everything, including errors, and bury the actual status in the response body. This breaks HTTP semantics, confuses clients, and makes monitoring harder. Use the right code:
| Code | When to Use |
|---|---|
200 OK | Successful GET, PATCH, DELETE |
201 Created | Successful POST that creates a resource |
204 No Content | Successful DELETE with no body |
400 Bad Request | Validation errors; malformed request body |
401 Unauthorized | Missing or invalid authentication token |
403 Forbidden | Authenticated but lacks permission |
404 Not Found | Resource does not exist |
409 Conflict | Resource already exists (duplicate email, etc.) |
422 Unprocessable Entity | Request is well-formed but semantically invalid |
429 Too Many Requests | Rate limit exceeded |
500 Internal Server Error | Unhandled server-side error |
Pagination, Filtering, and Sorting
Any endpoint that returns a list must support pagination. Without it, a single query can return millions of rows, crashing your server and giving your database administrator a very bad day.
// Cursor-based pagination (preferred for large, frequently updated datasets)
GET /api/v1/orders?cursor=eyJpZCI6MTAwfQ&limit=20
// Offset-based pagination (simpler; fine for smaller datasets)
GET /api/v1/products?page=3&limit=20
// Filtering and sorting
GET /api/v1/products?category=electronics&minPrice=50&maxPrice=500&sort=price:asc
// Response includes pagination metadata
{
"data": [...],
"meta": {
"page": 3,
"limit": 20,
"total": 1240,
"hasNextPage": true,
"nextCursor": "eyJpZCI6MTIwfQ"
}
}
Cursor-based pagination is more stable than offset-based when data is frequently inserted or deleted. If a new record is added between page 1 and page 2, offset pagination will skip or duplicate a record. Cursors don't have this problem.
Input Validation with DTOs
Never trust input from the client. Validate everything at the boundary of your system: in the controller layer, before the data ever reaches your services. Data Transfer Objects (DTOs) with validation decorators are the cleanest way to do this:
// create-user.dto.ts
import { IsEmail, IsString, MinLength, MaxLength, IsEnum } from 'class-validator';
export class CreateUserDto {
@IsEmail()
email: string;
@IsString()
@MinLength(8, { message: 'Password must be at least 8 characters' })
@MaxLength(100)
password: string;
@IsString()
@MinLength(2)
@MaxLength(50)
firstName: string;
@IsEnum(['customer', 'vendor'], { message: 'Role must be customer or vendor' })
role: 'customer' | 'vendor';
}
// user.controller.ts
@Post('/users')
async createUser(@Body() dto: CreateUserDto) {
// If validation fails, NestJS automatically returns 400 with field-level errors
// If we reach here, dto is guaranteed to be valid
return this.userService.create(dto);
}
Bonus Section B: Architectural Error Handling
Error handling is the part of architecture that most tutorials skip: and that most applications get badly wrong. Ad-hoc error handling leads to inconsistent responses, swallowed exceptions, and debugging sessions that feel like archaeology. A well-designed error handling architecture is centralized, consistent, and informative.
The Problem with Ad-Hoc Error Handling
// ❌ Error handling scattered everywhere: inconsistent responses
app.get('/users/:id', async (req, res) => {
try {
const user = await db.query('SELECT * FROM users WHERE id = ?', [req.params.id]);
if (!user) return res.status(404).json({ message: 'not found' });
res.json(user);
} catch (err) {
console.log(err);
res.status(500).json({ error: err.message }); // exposes internal details!
}
});
app.post('/orders', async (req, res) => {
try {
// ...
} catch (err) {
res.status(500).send('Something broke'); // completely different format
}
});
The result: every route handles errors differently, internal error messages (including stack traces in some cases) leak to clients, and there's no central place to add logging.
The Right Approach: Custom Exception Classes + Global Handler
The solution is a hierarchy of typed exceptions and a single global error handler that knows how to translate them into consistent HTTP responses.
// exceptions/base.exception.ts
export class AppException extends Error {
constructor(
public readonly message: string,
public readonly statusCode: number,
public readonly code: string,
) {
super(message);
this.name = this.constructor.name;
}
}
// exceptions/not-found.exception.ts
export class NotFoundException extends AppException {
constructor(resource: string) {
super(`${resource} not found`, 404, 'NOT_FOUND');
}
}
// exceptions/validation.exception.ts
export class ValidationException extends AppException {
constructor(public readonly details: Record<string, string>[]) {
super('Validation failed', 400, 'VALIDATION_ERROR');
}
}
// exceptions/forbidden.exception.ts
export class ForbiddenException extends AppException {
constructor(action?: string) {
super(action ? `Forbidden: ${action}` : 'Forbidden', 403, 'FORBIDDEN');
}
}
// middleware/error-handler.middleware.ts
export function globalErrorHandler(
err: Error,
req: Request,
res: Response,
next: NextFunction,
) {
const requestId = req.headers['x-request-id'] as string;
if (err instanceof AppException) {
logger.warn({ requestId, code: err.code, message: err.message });
return res.status(err.statusCode).json({
success: false,
error: { code: err.code, message: err.message },
requestId,
});
}
// Unexpected error: log full details internally, return generic message externally
logger.error({ requestId, error: err.stack });
return res.status(500).json({
success: false,
error: { code: 'INTERNAL_ERROR', message: 'An unexpected error occurred.' },
requestId,
});
}
// Now in services, simply throw typed exceptions:
async findUserById(id: string): Promise<User> {
const user = await this.userRepository.findById(id);
if (!user) throw new NotFoundException('User');
return user;
}
With this pattern, every error in the system produces a consistent, structured response. Internal error details never leak to clients. All errors are logged centrally. And adding a new error type means creating one small class: no changes to controllers or routes.
Domain vs. Infrastructure Errors
It's important to distinguish between two categories of errors:
- Domain errors: Expected business rule violations: "User not found," "Insufficient stock," "Order already cancelled." These are not bugs. They should be thrown as typed exceptions and return 4xx status codes.
- Infrastructure errors: Unexpected failures: database connection lost, third-party API timed out, disk full. These should be caught, logged with full context (stack trace, request ID, user ID), and translated into a generic 500 response. Never expose raw database or network errors to clients.
Bonus Section C: Logging, Monitoring, and Observability
You cannot maintain what you cannot see. Observability: the ability to understand the internal state of your system from its external outputs: is not an afterthought. It's a first-class architectural concern. Many teams only think about logging when something is on fire at 2 AM. By then, it's too late to instrument it properly.
Structured Logging
Plain-text log lines are fine for reading in a terminal. They're terrible for searching at scale. Structured logs emit JSON objects, which are machine-readable and can be indexed and queried in log management systems like Datadog, Grafana Loki, or AWS CloudWatch Insights.
// ❌ Unstructured logging: hard to query
console.log(`User ${userId} placed order ${orderId} for $${total}`);
// ✅ Structured logging: machine-readable, queryable
logger.info({
event: 'order.created',
userId,
orderId,
total,
currency: 'USD',
requestId: req.requestId,
durationMs: Date.now() - req.startTime,
});
// You can now query: event="order.created" AND total > 1000
// or: event="order.created" AND userId="usr_abc123"
Request Tracing with Correlation IDs
Assign a unique ID to every incoming request and attach it to every log line produced during that request's lifecycle. This allows you to reconstruct the complete journey of a single request through your system: across multiple services, workers, and database queries: in seconds.
// middleware/request-id.middleware.ts
import { v4 as uuidv4 } from 'uuid';
export function requestIdMiddleware(req: Request, res: Response, next: NextFunction) {
const requestId = (req.headers['x-request-id'] as string) || uuidv4();
req.requestId = requestId;
req.startTime = Date.now();
res.setHeader('x-request-id', requestId);
next();
}
// Every subsequent log in this request's lifecycle includes requestId:
logger.info({ requestId: req.requestId, event: 'user.login', userId });
logger.warn({ requestId: req.requestId, event: 'payment.retry', attempt: 2 });
What to Log and When
| Log Level | When to Use | Examples |
|---|---|---|
| DEBUG | Detailed info useful during development | Variable values, SQL queries, cache hits/misses |
| INFO | Normal, noteworthy business events | User registered, order placed, payment processed |
| WARN | Unexpected situations that didn't cause a failure | Rate limit approaching, deprecated API called, retry attempted |
| ERROR | Failures that need attention | Payment failed, email not sent, third-party API error |
| FATAL | System is in an unrecoverable state | Database connection permanently lost, out of memory |
Never log sensitive data. Passwords, payment card numbers, social security numbers, and authentication tokens must never appear in log output: even at DEBUG level. Mask or omit them before logging request bodies.
Application Metrics and Alerting
Beyond logs, expose metrics that allow you to monitor system health at a glance and alert before users notice problems. Essential metrics for any web application:
- Request rate: How many requests per second is each endpoint receiving?
- Error rate: What percentage of requests are returning 4xx or 5xx?
- Response time (p95, p99): Not just average: the 95th and 99th percentile latencies reveal tail latency problems the average hides.
- Database query time: Slow queries are the most common cause of sudden performance regressions.
- Queue depth: Is your job queue growing faster than workers are consuming it?
- Cache hit rate: A sudden drop in cache hit rate can reveal a cache invalidation bug or expiry misconfiguration.
Tools like Prometheus + Grafana, Datadog APM, or New Relic can collect and visualize these metrics with minimal instrumentation. Set alerts on error rate spikes and response time degradation so you know about problems before your users do.
Bonus Section D: Architecting Authentication and Authorization
Authentication and authorization are two of the most security-critical parts of any web application: and two of the most frequently misarchitected. Getting them right from day one protects your users, your data, and your business.
Authentication vs Authorization
| Concept | Question It Answers | Example |
|---|---|---|
| Authentication | "Who are you?" | Verifying a JWT token and identifying the user |
| Authorization | "What are you allowed to do?" | Checking if the user has the admin role before allowing delete |
JWT-Based Authentication Architecture
JSON Web Tokens (JWTs) are the standard for stateless authentication in modern web applications. The token carries the user's identity and is verified on every request: no database lookup required.
// auth.service.ts
@Injectable()
export class AuthService {
constructor(
private readonly userRepository: UserRepository,
private readonly jwtService: JwtService,
) {}
async login(email: string, password: string): Promise<AuthTokens> {
const user = await this.userRepository.findByEmail(email);
if (!user) throw new UnauthorizedException('Invalid credentials');
const isValid = await bcrypt.compare(password, user.passwordHash);
if (!isValid) throw new UnauthorizedException('Invalid credentials');
return this.generateTokens(user);
}
private generateTokens(user: User): AuthTokens {
const payload = { sub: user.id, email: user.email, role: user.role };
return {
accessToken: this.jwtService.sign(payload, { expiresIn: '15m' }),
refreshToken: this.jwtService.sign(payload, { expiresIn: '7d' }),
};
}
}
// auth.guard.ts: applied to protected routes
@Injectable()
export class JwtAuthGuard implements CanActivate {
constructor(private readonly jwtService: JwtService) {}
canActivate(context: ExecutionContext): boolean {
const request = context.switchToHttp().getRequest();
const token = request.headers.authorization?.split(' ')[1];
if (!token) throw new UnauthorizedException('Missing token');
try {
request.user = this.jwtService.verify(token);
return true;
} catch {
throw new UnauthorizedException('Invalid or expired token');
}
}
}
Role-Based Access Control (RBAC)
For most applications, a role-based system is the right level of granularity. Users have roles (e.g., customer, vendor, admin), and roles have permissions:
// roles.decorator.ts
export const Roles = (...roles: string[]) => SetMetadata('roles', roles);
// roles.guard.ts
@Injectable()
export class RolesGuard implements CanActivate {
constructor(private reflector: Reflector) {}
canActivate(context: ExecutionContext): boolean {
const requiredRoles = this.reflector.getAllAndOverride<string[]>('roles', [
context.getHandler(),
context.getClass(),
]);
if (!requiredRoles) return true; // no roles required
const { user } = context.switchToHttp().getRequest();
if (!requiredRoles.includes(user.role)) {
throw new ForbiddenException('Insufficient permissions');
}
return true;
}
}
// Usage in controllers:
@Delete('/products/:id')
@UseGuards(JwtAuthGuard, RolesGuard)
@Roles('admin')
async deleteProduct(@Param('id') id: string) {
return this.productService.delete(id);
}
Security Best Practices for Authentication
- Short-lived access tokens: 15 minutes is a reasonable default. Short expiry limits the damage if a token is stolen.
- Refresh token rotation: Issue a new refresh token each time one is used, and invalidate the old one. This limits the window for stolen refresh tokens.
- Store refresh tokens in HTTP-only cookies: They cannot be accessed by JavaScript, protecting against XSS attacks.
- Hash passwords with bcrypt: Never store plaintext or MD5/SHA-hashed passwords. Use bcrypt with a cost factor of 12+.
- Rate limit login endpoints: Prevent brute-force attacks by limiting login attempts per IP and per account.
- Log authentication events: Log successful logins, failed attempts, and password changes with IP address and timestamp for audit purposes.
Bonus Section E: Database Design Principles for Scalable Applications
The database is typically the bottleneck in any web application. Poor database design: whether in schema structure, indexing, or query patterns: can undermine even the most beautifully architected application layer. Getting the fundamentals right from day one is essential.
Choosing the Right Database
| Database Type | Best For | Examples | Watch Out For |
|---|---|---|---|
| Relational (SQL) | Structured data with complex relationships; financial data; anything requiring ACID transactions | PostgreSQL, MySQL | Schema migrations on large tables; vertical scaling limits |
| Document (NoSQL) | Flexible schemas; hierarchical or embedded data; content management | MongoDB, DynamoDB | Complex joins; lack of ACID across documents; data duplication |
| Key-Value | Caching; sessions; real-time leaderboards; rate limiting | Redis, Memcached | Not suitable as primary data store; data loss on restart (without persistence) |
| Search Engine | Full-text search; autocomplete; faceted filtering | Elasticsearch, Meilisearch | Not a primary store; sync complexity; not for transactional data |
| Time-Series | Metrics; IoT data; analytics events | InfluxDB, TimescaleDB | Specialized; unnecessary overhead for general-purpose data |
For most web applications, PostgreSQL is the right default. It supports JSON columns for flexibility, has exceptional indexing capabilities, and can handle millions of rows on a single instance. Start there. Add specialized databases (Redis, Elasticsearch) when you have a specific, validated need.
Schema Design Principles
Use UUIDs for Primary Keys
Sequential integer IDs are predictable: they expose your data volume to competitors and are trivially guessable by attackers. UUIDs are opaque and globally unique, making them safe to expose in URLs and APIs. Use UUID v4 for random IDs or UUID v7 for sortable time-ordered IDs (better index performance).
-- PostgreSQL: UUID primary key with auto-generation
CREATE TABLE users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
email VARCHAR(255) NOT NULL UNIQUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
Always Include Timestamps
Every table should have created_at and updated_at columns. These are invaluable for debugging, auditing, analytics, and cache invalidation. Add them by default: you'll never regret it and you'll always regret not having them.
Soft Deletes for Important Data
For entities that users care about: orders, accounts, products: consider soft deletes instead of hard deletes. Add a deleted_at nullable timestamp column. Records with a non-null deleted_at are "deleted" in the application but remain in the database for auditing and recovery.
-- Soft delete column
ALTER TABLE orders ADD COLUMN deleted_at TIMESTAMPTZ;
-- Application query always filters deleted records
SELECT * FROM orders WHERE deleted_at IS NULL AND user_id = $1;
-- "Delete" is actually an update
UPDATE orders SET deleted_at = NOW() WHERE id = $1;
Indexing Strategy
Indexes are one of the highest-leverage performance tools you have: and one of the most commonly misused. Too few indexes means slow queries. Too many indexes means slow writes and bloated storage.
-- Always index foreign keys (they're almost always used in JOINs and WHERE clauses)
CREATE INDEX idx_orders_user_id ON orders(user_id);
CREATE INDEX idx_order_items_order_id ON order_items(order_id);
CREATE INDEX idx_order_items_product_id ON order_items(product_id);
-- Index columns used in WHERE clauses for frequent queries
CREATE INDEX idx_orders_status ON orders(status);
CREATE INDEX idx_users_email ON users(email); : also enforces uniqueness if UNIQUE
-- Composite index for multi-column queries (column order matters!)
-- Optimizes: WHERE user_id = $1 AND status = $2 ORDER BY created_at DESC
CREATE INDEX idx_orders_user_status_date ON orders(user_id, status, created_at DESC);
-- Partial index: only indexes rows matching a condition (saves space)
CREATE INDEX idx_orders_pending ON orders(created_at) WHERE status = 'pending';
Database Migrations
Schema changes must be version-controlled, reproducible, and reversible. Use a migration tool from day one: never manually alter a production database schema.
// Example using Prisma Migrate (Node.js ecosystem)
// 1. Update your schema.prisma file
model Product {
id String @id @default(uuid())
name String
price Int // in cents
categoryId String
isActive Boolean @default(true)
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
}
// 2. Generate and apply migration
// npx prisma migrate dev --name add_product_table
// Creates a timestamped migration file:
// migrations/20260120143022_add_product_table/migration.sql
// This file is committed to version control and applied in CI/CD
Zero-downtime migrations require care on large tables. Adding a NOT NULL column without a default, dropping a column, or renaming a column can lock the table and cause downtime. Use techniques like adding columns as nullable first, backfilling data in batches, then adding the NOT NULL constraint: or use tools like pg-osc (Online Schema Change for PostgreSQL) for large tables.
The N+1 Query Problem
The N+1 problem is one of the most common performance bugs in applications that use ORMs. It occurs when you fetch a list of N records and then make one additional database query for each record to fetch related data: resulting in N+1 total queries instead of 1 or 2.
// ❌ N+1 Problem: makes 1 + N database queries
const orders = await Order.findAll(); // 1 query, returns 100 orders
for (const order of orders) {
order.user = await User.findOne({ where: { id: order.userId } }); // 100 queries!
}
// Total: 101 database round trips
// ✅ Eager loading: makes 2 database queries regardless of result size
const orders = await Order.findAll({
include: [{ model: User, as: 'user' }]
});
// Total: 2 database round trips (orders + users joined/batched)
Bonus Section F: Building a Comprehensive Testing Strategy
Testing is not separate from architecture: it's a direct consequence of it. Well-architected code with proper separation of concerns and dependency injection is naturally testable. Poorly architected code makes testing so painful that teams often abandon it. The testing strategy you adopt should align with your architecture, not fight against it.
The Testing Pyramid
┌─────────────────┐
│ E2E Tests │ ← Slowest, fewest (~10)
│ (Full stack) │
├─────────────────┤
│ Integration │ ← Medium speed, moderate (~50-100)
│ Tests (API) │
├─────────────────┤
│ │
│ Unit Tests │ ← Fastest, most numerous (~500+)
│ (Services, │
│ Utilities) │
└─────────────────┘
Invest most heavily at the bottom of the pyramid. Unit tests are fast, isolated, and give the tightest feedback loop. Too many E2E tests make your test suite slow and brittle.
Unit Testing Services
Your service layer contains all business logic: it should have comprehensive unit test coverage. Because services use dependency injection, mocking dependencies is trivial:
// order.service.spec.ts
describe('OrderService', () => {
let orderService: OrderService;
let mockOrderRepository: jest.Mocked<IOrderRepository>;
let mockProductRepository: jest.Mocked<IProductRepository>;
let mockPaymentService: jest.Mocked<IPaymentService>;
beforeEach(() => {
mockOrderRepository = { create: jest.fn(), findById: jest.fn() } as any;
mockProductRepository = { findById: jest.fn() } as any;
mockPaymentService = { charge: jest.fn() } as any;
orderService = new OrderService(
mockOrderRepository,
mockProductRepository,
mockPaymentService,
);
});
describe('createOrder', () => {
it('should throw NotFoundException when product does not exist', async () => {
mockProductRepository.findById.mockResolvedValue(null);
await expect(
orderService.createOrder('usr_1', { productId: 'prod_999', quantity: 1 })
).rejects.toThrow(NotFoundException);
});
it('should throw BadRequestException when stock is insufficient', async () => {
mockProductRepository.findById.mockResolvedValue({
id: 'prod_1', price: 2999, stock: 0,
});
await expect(
orderService.createOrder('usr_1', { productId: 'prod_1', quantity: 1 })
).rejects.toThrow(BadRequestException);
});
it('should create order successfully when all conditions are met', async () => {
mockProductRepository.findById.mockResolvedValue({
id: 'prod_1', price: 2999, stock: 10,
});
mockPaymentService.charge.mockResolvedValue({ id: 'ch_test_123' });
mockOrderRepository.create.mockResolvedValue({ id: 'ord_1', total: 2999 });
const result = await orderService.createOrder('usr_1', {
productId: 'prod_1', quantity: 1,
});
expect(result.id).toBe('ord_1');
expect(mockPaymentService.charge).toHaveBeenCalledWith('usr_1', 2999);
});
});
});
Integration Testing API Endpoints
Integration tests spin up your actual application (or a close approximation of it) and test real HTTP interactions. They're slower than unit tests but catch issues that unit tests miss: middleware, routing, serialization, and the interaction between layers:
// orders.integration.spec.ts
describe('Orders API', () => {
let app: INestApplication;
beforeAll(async () => {
const moduleRef = await Test.createTestingModule({
imports: [AppModule],
})
.overrideProvider(DatabaseService)
.useValue(testDatabaseService) // use a test database
.compile();
app = moduleRef.createNestApplication();
await app.init();
});
it('POST /api/v1/orders should return 401 without token', async () => {
return request(app.getHttpServer())
.post('/api/v1/orders')
.send({ productId: 'prod_1', quantity: 2 })
.expect(401);
});
it('POST /api/v1/orders should create order for authenticated user', async () => {
const token = await getTestAuthToken(app, testUser);
return request(app.getHttpServer())
.post('/api/v1/orders')
.set('Authorization', `Bearer ${token}`)
.send({ productId: seedProduct.id, quantity: 1 })
.expect(201)
.expect(res => {
expect(res.body.data.id).toBeDefined();
expect(res.body.data.total).toBe(seedProduct.price);
});
});
});
What to Test and What to Skip
| Test This | Consider Skipping |
|---|---|
| Business logic in services (all branches and edge cases) | Simple getters and setters with no logic |
| Authentication and authorization rules | Framework boilerplate (routing decorators, module wiring) |
| Input validation on DTOs | Third-party library behavior (test your usage of it, not the library) |
| Error handling paths (what happens when things fail) | One-line utility functions with no branching |
| Complex data transformations and calculations | Auto-generated code (ORM models, proto files) |
| Critical API endpoints (login, payment, order creation) | Static configuration files |
Bonus Section G: Environment Configuration and Secrets Management
Mismanaged configuration is a silent killer of both security and deployment reliability. Hard-coded database URLs, API keys checked into version control, and environment-specific logic scattered through the codebase are among the most common architectural hygiene problems. A clean configuration strategy from day one prevents all of these.
The 12-Factor App: Config Principle
The 12-Factor App methodology states that everything that varies between deployment environments (development, staging, production) should be stored in environment variables: not in the code, not in config files committed to version control.
// ❌ Hard-coded configuration: never do this
const db = new Database({
host: 'prod-db.company.internal',
password: 'super_secret_password_123', // visible to everyone with repo access!
});
// ✅ Environment-variable driven configuration
const db = new Database({
host: process.env.DATABASE_HOST,
password: process.env.DATABASE_PASSWORD,
});
A Typed Configuration Service
Rather than calling process.env.SOME_VAR scattered throughout the codebase, centralize all configuration in a typed service that validates required variables at startup:
// config/env.ts: validates and types all environment variables at startup
import { z } from 'zod';
const envSchema = z.object({
NODE_ENV: z.enum(['development', 'staging', 'production']),
PORT: z.coerce.number().default(3000),
DATABASE_URL: z.string().url(),
DATABASE_POOL_SIZE: z.coerce.number().default(10),
REDIS_URL: z.string().url(),
JWT_SECRET: z.string().min(32),
JWT_EXPIRES_IN: z.string().default('15m'),
STRIPE_SECRET_KEY: z.string().startsWith('sk_'),
STRIPE_WEBHOOK_SECRET: z.string().startsWith('whsec_'),
SMTP_HOST: z.string(),
SMTP_PORT: z.coerce.number(),
SMTP_USER: z.string().email(),
SMTP_PASSWORD: z.string(),
AWS_REGION: z.string().optional(),
AWS_S3_BUCKET: z.string().optional(),
});
// Parse and validate at startup: if any required variable is missing,
// the application refuses to start with a clear error message
const parsed = envSchema.safeParse(process.env);
if (!parsed.success) {
console.error('❌ Invalid environment configuration:');
console.error(parsed.error.format());
process.exit(1); // Fail fast: don't start with bad config
}
export const config = parsed.data;
This approach means your application fails fast at startup if any required configuration is missing, rather than failing mysteriously at runtime when the missing variable is first accessed. It also gives every team member a single, documented source of truth for what the application needs to run.
Managing Secrets in Production
In production, environment variables should not be stored in files or set manually. Use a dedicated secrets management solution:
- AWS Secrets Manager / Parameter Store: Inject secrets at container startup; supports automatic rotation.
- HashiCorp Vault: Self-hosted, highly flexible secrets engine with dynamic secrets and fine-grained access control.
- Doppler / Infisical: Developer-friendly secrets managers with CI/CD integrations and team access controls.
- Kubernetes Secrets: For Kubernetes deployments; integrate with external secret stores for production.
Never commit secrets to version control: not even in private repositories. Rotate any secret that has been committed immediately. Add .env to your .gitignore on the very first commit and provide a .env.example file with all required variables (but no real values) as documentation.
Bonus Section H: CI/CD and Deployment Architecture
Your deployment process is as much a part of your application architecture as your folder structure. A robust CI/CD pipeline is what turns a well-written application into a reliably delivered product. Without it, every deployment is a manual, error-prone ritual that slows the team down and introduces risk.
The Deployment Pipeline
Developer pushes code
↓
CI Pipeline triggered (GitHub Actions / GitLab CI / Jenkins)
↓
┌─────────────────────────────────────┐
│ Stage 1: Build & Lint │
│ - Install dependencies │
│ - TypeScript compilation check │
│ - ESLint / code style check │
└────────────────┬────────────────────┘
↓
┌─────────────────────────────────────┐
│ Stage 2: Test │
│ - Run unit tests │
│ - Run integration tests │
│ - Generate coverage report │
│ - Fail if coverage drops below 80% │
└────────────────┬────────────────────┘
↓
┌─────────────────────────────────────┐
│ Stage 3: Security Scan │
│ - npm audit (dependency vulns) │
│ - SAST scan (Snyk / Semgrep) │
│ - Secrets scan (truffleHog) │
└────────────────┬────────────────────┘
↓
┌─────────────────────────────────────┐
│ Stage 4: Build Docker Image │
│ - Multi-stage Dockerfile │
│ - Push to container registry │
│ - Tag with git commit SHA │
└────────────────┬────────────────────┘
↓
Deploy to Staging (automatic)
↓
Run E2E Tests against staging
↓
Deploy to Production (manual gate
or automatic on main branch)
Dockerfile Best Practices
# Multi-stage build: keeps production image small and secure
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production # install only production deps
COPY . .
RUN npm run build # compile TypeScript
# Production image: only what's needed to run
FROM node:20-alpine AS production
WORKDIR /app
RUN addgroup -S appgroup && adduser -S appuser -G appgroup # non-root user
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./
USER appuser # never run as root in production
EXPOSE 3000
CMD ["node", "dist/main.js"]
Zero-Downtime Deployments
Your deployment strategy should ensure that users never experience downtime during a release. The key techniques:
- Rolling deployments: Gradually replace old instances with new ones. At any point during deployment, some instances run the old version and some run the new version. Requires that both versions are compatible simultaneously.
- Blue-Green deployments: Maintain two identical environments (blue = current production, green = new version). Switch traffic from blue to green when the new version is validated. Instant rollback by switching back to blue.
- Feature flags: Deploy new code to production but hide it behind a feature flag. Enable for internal users first, then gradually roll out to users. Separate deployment from release.
- Database migration strategy: Apply backward-compatible database migrations before deploying new application code. New code must work with both old and new schema during the transition period.
Further Reading and Resources
The principles in this guide draw from decades of accumulated software engineering wisdom. If you want to go deeper, these are the resources that have most shaped how experienced architects think about large-scale software design:
Essential Books
- Clean Architecture by Robert C. Martin: The definitive treatment of layered architecture and the dependency rule. Required reading for any serious software engineer.
- Designing Data-Intensive Applications by Martin Kleppmann: The best book available on database internals, distributed systems, and scalability. Dense but invaluable.
- Domain-Driven Design by Eric Evans: The source of many concepts used in modern enterprise architecture: bounded contexts, aggregates, repositories, and ubiquitous language.
- The Pragmatic Programmer by Hunt and Thomas: Timeless practical wisdom on software craftsmanship. Read it once a year.
- Software Architecture Patterns by Mark Richards: A short, practical overview of the most common architectural styles: layered, event-driven, microservices, space-based, and more.
- Building Microservices by Sam Newman: The go-to reference when you're genuinely ready to move beyond a monolith.
Key Concepts to Explore Next
- Domain-Driven Design (DDD): Bounded contexts, aggregates, and value objects are powerful tools for managing complexity in large systems.
- Event Sourcing and CQRS: Advanced patterns for systems that need a complete audit trail or need to separate read and write models for scalability.
- The Twelve-Factor App (12factor.net): A methodology for building software-as-a-service apps that are portable, deployable, and scalable.
- OpenAPI / Swagger: Document your API as you build it. Auto-generated documentation that stays in sync with your code is one of the highest-ROI investments in developer experience.
- OpenTelemetry: The emerging standard for distributed tracing, metrics, and logging across polyglot microservices environments.
Architecture is a discipline you deepen over years, not weeks. Every project you build teaches you something. The engineers who grow fastest are those who reflect deliberately on what worked, what didn't, and why: and who seek out the principles that explain those observations.