Today I focused on building the async backbone of my media processing system using NestJS + BullMQ + Redis. Implemented an upload confirmation pipeline with a producer–consumer architecture that separates API traffic from heavy background processing. Set up Redis in Docker, built a scalable job queue, and split the system into API and worker apps using a clean, modular architecture. Big takeaway: designing for async from day one makes systems more resilient, scalable, and production-ready. Step by step turning this into a cloud-native media pipeline 🚀 #NestJS #BackendEngineering #SystemDesign #Redis #AsyncArchitecture https://lnkd.in/giE46bsx
Nahid Hasan’s Post
More Relevant Posts
-
transforming from monolith to micro-services with K8s, 😄 in monolith services, when one component breaks, the whole system breaks, but for micro-services, that is not true. There, the components are all separated, and they do run separately......... Here, I have a github, in which i have shared how I changed my whiteboard application from monolith to microservices. I used Kubernetes | Kind cluster https://lnkd.in/gpb3hbNH
To view or add a comment, sign in
-
🚀 Just shipped a production-grade, fully serverless portfolio web app with multi-user OAuth 2.0 (PKCE) — built to reflect real-world AWS architecture, security, and IaC best practices. Live: https://lnkd.in/gnZydNQk Code: https://lnkd.in/gPnv7NaF What I built A multi-user web application that lets visitors securely authenticate with Spotify and get personalized insights—without exposing client secrets. ✅ OAuth 2.0 PKCE (browser-based) ✅ AWS Lambda + API Gateway + DynamoDB backend ✅ CloudFront + S3 frontend with Origin Access Control (no public bucket access) ✅ KMS encryption for tokens and sensitive data at rest ✅ EventBridge scheduling for automated pipelines ✅ 100% Terraform, modular and reusable ✅ Cost-optimized: ~$4–5/month Architecture (high level) CloudFront → S3 (static frontend) API Gateway → Lambda → DynamoDB / Secrets Manager / KMS (secure backend + storage) By the numbers 2,232 lines of Python in the Lambda backend 15+ API endpoints 6 DynamoDB tables (users, sessions, tokens, insights, etc.) 8 Terraform modules 44+ AWS resources under Terraform Session strategy: 24-hour visitor sessions + long-lived admin session (managed securely) Key engineering decisions PKCE to prevent authorization code interception and eliminate client secret exposure Tokens encrypted via AWS KMS (AES-256) before storage DynamoDB TTL + caching to reduce external API calls and control cost Module-level Lambda clients to reduce cold-start overhead Least-privilege IAM policies and policy-driven deployments Discussion If you’ve run serverless in production: what patterns have been most effective for security, cost control, and observability? #CloudEngineering #AWS #Serverless #Terraform #InfrastructureAsCode #DevOps #CloudArchitecture #AWSLambda #DynamoDB #APIGateway #CloudFront #OAuth #CloudSecurity #PlatformEngineering #SRE #BuildInPublic
To view or add a comment, sign in
-
From “how do I run NestJS and Prisma in Lambda?” to a full monorepo architecture. I wanted a serverless backend that felt like a normal NestJS + Prisma app: shared types, shared services, one codebase, but each “service” as its own Lambda. So I built it: https://lnkd.in/dCAPzW6A What’s in place: → Monorepo with shared packages (types, utils, constants, services) used by every Lambda → Lambda layers for NestJS, Prisma, auth, and AWS SDK so functions stay small and cold starts reasonable → AWS CDK with NodejsFunction: esbuild bundles each Lambda from source using path aliases—no pre-building shared packages for deploy → API Gateway REST API, env-specific config, and GitHub Actions for CI and deploy (dev/main → dev/prod) The goal was: add a Lambda by adding a folder and wiring it in the stack—no custom bundling scripts, no fighting the framework. It’s there. If you’re going down the “NestJS + Prisma in Lambda” or “multi-Lambda monorepo” path, I’m happy to share what worked and what I’d do differently. #AWS #Serverless #NestJS #Prisma #CDK #Monorepo #Backend #TypeScript #Architecture
To view or add a comment, sign in
-
🚀 Excited to Share My Users Microservice (TypeScript + MongoDB) I’ve built a production-ready Users Microservice in TypeScript that you can plug into any project — so you don’t have to handle user management yourself. Just focus on building your core features. ✨ Key Features Sign up & Login Sign in with Google Update password Forgot password with OTP Hashing password Reset password Access & Refresh tokens Secure token handling via cookies MongoDB integration Graceful startup & shutdown (production-ready) Winston logger integrated with AWS CloudWatch 🔍 Actuator Endpoints (Infra Monitoring) /actuator/health → Real DB + service health status /actuator/liveness → Process health (K8s liveness probe) /actuator/readiness → Traffic readiness (K8s readiness probe) /actuator/system → OS + memory breakdown (Admin access only) The repository includes: Clean folder structure Well-defined routes Detailed README.md .env.example for easy environment setup This service is designed with scalability, monitoring, and real-world production needs in mind. If you're building microservices or scaling your backend, this might save you a lot of time. 🔗 Check out the repository link below and explore the full documentation. https://lnkd.in/g37QAhTD #microservice #backend #nodejs #ai #agents #users #service Would love your feedback and thoughts!
To view or add a comment, sign in
-
If you need quick and dirty work like a small API, a script, or glue code, Node.js is still very good. Keep it simple and use TypeScript if you want type checking. Pick one small web framework like Express or Fastify and try hard to NOT add lots of extra packages. A big part of 'Node breaks' pain is dependency bloat. If you are building a bigger backend service that must run calm in production, Go is a strong default, obviously Go gives real types, fast builds, and one compiled binary you can ship anywhere. It also makes Docker simpler. A basic Go program that imports net/http is about 10 MB, and many real microservices land around 13 MB to 17 MB unstripped. That is small enough that image size stops being a problem. For containers, use a multi-stage Docker build and copy the final Go binary into a minimal runtime image. Alpine is fine. If you want the smallest image and you do not need a shell, use scratch. For microservices support, Go tends to feel better because you can get common service needs wired up faster. If you want batteries included patterns, look at go-zero https://lnkd.in/g_qMD9TY If you want cross language building blocks like service discovery, tracing, and logging components, look at Dapr (dapr/dapr). Though do not expect Go to be perfect, if your stack uses MongoDB, then be ready for friction with the official Go MongoDB driver. Some teams choose a third party library because they dislike the official driver APIs (including me). Also, editor support matters. If VSCode Go IntelliSense feels flaky, try GoLand. It is paid, has a 30 day trial, and students can get it free
To view or add a comment, sign in
-
Setting up your own container registry is needed in some architectures. Doing this often means choosing between heavyweight enterprise tools that eat resources and budget, or going without a UI altogether. The article below describes a practical open-source alternative that gives DevOps teams image visibility and control with a minimal footprint and zero licensing costs. The example shows a full stack setup, using a FastAPI backend and a React frontend with Docker Compose orchestration. Real deployment steps for single-server, multi-server, and cloud setups are included. Production hardening, backup strategies, and CI/CD integration are also discussed. This is a good guide from Madhu Sudhan Reddy Y.B that can get you up and running with your own self-hosted registries. Check it out! https://lnkd.in/eavfa_V9
To view or add a comment, sign in
-
A production ready docker registry UI - alternate for Jfrog artifactory and Amazon ECR for small teams - to avoid high costs
Setting up your own container registry is needed in some architectures. Doing this often means choosing between heavyweight enterprise tools that eat resources and budget, or going without a UI altogether. The article below describes a practical open-source alternative that gives DevOps teams image visibility and control with a minimal footprint and zero licensing costs. The example shows a full stack setup, using a FastAPI backend and a React frontend with Docker Compose orchestration. Real deployment steps for single-server, multi-server, and cloud setups are included. Production hardening, backup strategies, and CI/CD integration are also discussed. This is a good guide from Madhu Sudhan Reddy Y.B that can get you up and running with your own self-hosted registries. Check it out! https://lnkd.in/eavfa_V9
To view or add a comment, sign in
-
I am excited to share one of my latest projects, StartTech Infra, where I architected and deployed a fully automated, self-healing cloud infrastructure for a 3-tier web application! The Challenge The goal was to take a monolithic application and transform it into a scalable, cloud-native solution. I needed to ensure zero-downtime deployments, robust security, and complete observability without manual intervention. The Solution I built a 3-Tier Architecture using Terraform for Infrastructure as Code (IaC) and GitHub Actions for CI/CD. Tech Stack Infrastructure: AWS (EC2, ALB, ASG, S3, CloudFront, MongoDB Atlas, ElastiCache Redis) IaC: Terraform (Remote State w/ Locking) CI/CD: GitHub Actions Backend: Go (Golang) Frontend: React (Vite) Key Features implemented Immutable Infrastructure: Replaced mutable updates with ASG Instance Refresh, treating servers as disposable to eliminate configuration drift. Automated CI/CD Pipelines: Frontend: S3 sync + CloudFront Invalidation. Backend: Docker build -> ECR push -> ASG Rotation. Security: Embedded trivy and tfsec scanning in every PR. High Availability: Multi-AZ deployment with Application Load Balancer and Auto Scaling (Self-healing). Secret Management: strict separation of secrets using GitHub Actions Secrets and Terraform variables (no hardcoded keys!). Results Deployment Time: Reduced from manual steps to <5 minutes per commit. Reliability: 99.9% uptime with automated health checks. Security: 100% compliance with least-privilege IAM roles. Check out the code! Infrastructure Repo: https://lnkd.in/dNNQtRf6 Application Repo: https://lnkd.in/d9_xVKtQ #DevOps #AWS #Terraform #Golang #CloudComputing #CICD #GitHubActions #InfrastructureAsCode #CloudNative #altschoolafrica
To view or add a comment, sign in
-
Excited to share a screen walkthrough of my portfolio — and the centerpiece: Krafti Vibe, a multi-tenant SaaS platform I've been architecting from the ground up since 2023. Krafti Vibe is built to empower artisans and small-scale creators with secure, scalable tools to manage their businesses — and the backend reflects that ambition: ⚙️ Multi-tenant architecture with PostgreSQL Row-Level Security (RLS) for airtight data isolation 🔐 Secure auth via OAuth2/OIDC (Zitadel) + JWT-protected APIs 🚀 High-performance Go backend with GORM, Redis caching & clean architecture 📦 Containerized with Docker, orchestrated on Kubernetes, and fully automated via GitHub Actions CI/CD 📊 Observability stack with Prometheus & Grafana ⚡ Sub-100ms API response times, WebSocket support for real-time features, and 100% on-time milestone delivery With 132+ commits and counting, the project continues to evolve — shaped by Domain-Driven Design, microservices patterns, and a commitment to scalability and security from day one. If you're building in the multi-tenant SaaS, Golang, PostgreSQL RLS, or Kubernetes space — I'd love to connect and exchange insights. 🔗 Portfolio: peridev.org 💻 GitHub: https://lnkd.in/d8QEkYGX #Golang #BackendDevelopment #SaaS #MultiTenant #PostgreSQL #Kubernetes #DevOps #SoftwareEngineering
To view or add a comment, sign in
-
How to Reduce Docker Image Size & Use Fewer Resources : One thing I’ve learned while working with containers: Smaller Docker images = Faster builds + Faster deployments + Lower cloud cost. Here are some practical ways to reduce Docker image size: 1. Use Lightweight Base Images Instead of: FROM node:18 Use: FROM node:18-alpine Alpine images are much smaller because they use a minimal Linux distribution. 2. Use Multi-Stage Builds Multi-stage builds help you separate build dependencies from production runtime. # Stage 1 - Build FROM node:18-alpine AS builder WORKDIR /app COPY package*.json . RUN npm install COPY . . RUN npm run build # Stage 2 - Production FROM node:18-alpine WORKDIR /app COPY --from=builder /app/dist ./dist COPY package*.json . RUN npm install --only=production CMD ["node", "dist/server.js"] This removes unnecessary dev dependencies from the final image. 3. Use .dockerignore Prevent unnecessary files from being copied into the image: node_modules .git .env logs This keeps your image clean and small. 4. Install Only Production Dependencies Instead of: npm install Use: npm install --only=production Removes dev dependencies in production builds. 5. Reduce Layers Combine commands: Bad: RUN apt-get update RUN apt-get install -y curl Better: RUN apt-get update && apt-get install -y curl Fewer layers = smaller image. 6. Clean Cache Files RUN apt-get update && apt-get install -y curl \ && rm -rf /var/lib/apt/lists/* Always remove unnecessary package cache. Why This Matters: • Faster CI/CD pipelines • Lower storage cost • Better performance • Reduced attack surface • More scalable microservices Optimizing Docker images may look small, but in large-scale systems, it makes a huge difference. #Aaptor #Docker #DevOps #Backend #Cloud #SoftwareEngineering #NodeJS #MERN #AWS
To view or add a comment, sign in
-
Explore content categories
- Career
- Productivity
- Finance
- Soft Skills & Emotional Intelligence
- Project Management
- Education
- Technology
- Leadership
- Ecommerce
- User Experience
- Recruitment & HR
- Customer Experience
- Real Estate
- Marketing
- Sales
- Retail & Merchandising
- Science
- Supply Chain Management
- Future Of Work
- Consulting
- Writing
- Economics
- Artificial Intelligence
- Employee Experience
- Workplace Trends
- Fundraising
- Networking
- Corporate Social Responsibility
- Negotiation
- Communication
- Engineering
- Hospitality & Tourism
- Business Strategy
- Change Management
- Organizational Culture
- Design
- Innovation
- Event Planning
- Training & Development