ByteByteGo

I’ve been writing the system design newsletter for 12 months. Here are the 5 most popular ones: 👇 1. From 0 to Millions: A Guide to Scaling Your App 2. A Crash Course in Caching 3. API Architectural Styles 4. How does ChatGPT work? 5. 8 Data Structures That Power Your Databases Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3FEGliw .

The Big Archive for System Design - 2023 Edition (PDF) is available now. And it's completely FREE. The PDF contains 𝐚𝐥𝐥 𝐦𝐲 𝐭𝐞𝐜𝐡𝐧𝐢𝐜𝐚𝐥 𝐩𝐨𝐬𝐭𝐬 published in 2023. What’s included in the PDF? 🔹 Netflix's Tech Stack 🔹 Top 5 common ways to improve API performance 🔹 Linux boot Process Explained 🔹 CAP, BASE, SOLID, KISS, What do these acronyms mean? 🔹 Explaining JSON Web Token (JWT) to a 10 year old Kid 🔹 Explaining 8 Popular Network Protocols in 1 Diagram 🔹 Top 5 Software Architectural Patterns 🔹 OAuth 2.0 Flows 🔹 What does API gateway do? 🔹 Linux file system explained 🔹 18 Key Design Patterns Every Developer Should Know 🔹 Best ways to test system functionality 🔹 Top 6 Load Balancing Algorithms 🔹 Top 12 Tips for API Security 🔹 𝐀𝐧𝐝 100+ 𝐦𝐨𝐫𝐞 Like, follow and subscribe to our newsletter to receive the PDF download link: https://bit.ly/3KCnWXq #systemdesign #coding #interviewtips .

How do DevOps, NoOps change the software development lifecycle (SDLC)? The diagram below compares traditional SDLC, DevOps and NoOps. In a traditional software development, code, build, test, release and monitoring are siloed functions. Each stage works independently and hands over to the next stage. DevOps, on the other hand, encourages continuous development and collaboration between developers and operations. This shortens the overall life cycle and provides continuous software delivery. NoOps is a newer concept with the development of serverless computing. Since we can architect the system using FaaS (Function-as-a-Service) and BaaS (Backend-as-a-Service), the cloud service providers can take care of most operations tasks. The developers can focus on feature development and automate operations tasks. NoOps is a pragmatic and effective methodology for startups or smaller-scale applications, which moves shortens the SDLC even more than DevOps. –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3KCnWXq    #systemdesign #coding #interviewtips  .

Life is Short, Use Dev Tools    The right dev tool can save you precious time, energy, and perhaps the weekend as well.    Here are our favorite dev tools:    1 - Development Environment  A good local dev environment is a force multiplier. Powerful IDEs like VSCode, IntelliJ IDEA, Notepad++, Vim, PyCharm & Jupyter Notebook can make your life easy.    2 - Diagramming  Showcase your ideas visually with diagramming tools like DrawIO, Excalidraw, mindmap, Mermaid, PlantUML, Microsoft Visio, and Miro    3 - AI Tools  AI can boost your productivity. Don’t ignore tools like ChatGPT, GitHub Copilot, Tabnine, Claude, Ollama, Midjourney, and Stable Diffusion.    4 - Hosting and Deployment  For hosting your applications, explore solutions like AWS, Cloudflare, GitHub, Fly, Heroku, and Digital Ocean.    5 - Code Quality  Quality code is a great differentiator. Leverage tools like Jest, ESLint, Selenium, SonarQube, FindBugs, and Checkstyle to ensure top-notch quality.    6 - Security  Don’t ignore the security aspects and use solutions like 1Password, LastPass, OWASP, Snyk, and Nmap.    7 - Note-taking  Your notes are a reflection of your knowledge. Streamline your note-taking with Notion, Markdown, Obsidian, Roam, Logseq, and Tiddly Wiki.    8 - Design  Elevate your visual game with design tools like Figma, Sketch, Adobe Illustrator, Canva, and Adobe Photoshop.    Over to you: Which dev tools do you use? –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3KCnWXq    #systemdesign #coding #interviewtips  .

Top 9 Architectural Patterns for Data and Communication Flow  .  .  🔹 Peer-to-Peer  The Peer-to-Peer pattern involves direct communication between two components without the need for a central coordinator.    🔹 API Gateway  An API Gateway acts as a single entry point for all client requests to the backend services of an application.    🔹 Pub-Sub  The Pub-Sub pattern decouples the producers of messages (publishers) from the consumers of messages (subscribers) through a message broker.    🔹 Request-Response  This is one of the most fundamental integration patterns, where a client sends a request to a server and waits for a response.    🔹 Event Sourcing  Event Sourcing involves storing the state changes of an application as a sequence of events.    🔹 ETL  ETL is a data integration pattern used to gather data from multiple sources, transform it into a structured format, and load it into a destination database.    🔹 Batching  Batching involves accumulating data over a period or until a certain threshold is met before processing it as a single group.    🔹 Streaming Processing  Streaming Processing allows for the continuous ingestion, processing, and analysis of data streams in real-time.    🔹 Orchestration  Orchestration involves a central coordinator (an orchestrator) managing the interactions between distributed components or services to achieve a workflow or business process. –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3KCnWXq    #systemdesign #coding #interviewtips  .

How is data sent over the internet? What does that have to do with the OSI model? How does TCP/IP fit into this?    7 Layers in the OSI model are:  1. Physical Layer  2. Data Link Layer  3. Network Layer  4. Transport Layer  5. Session Layer  6. Presentation Layer  7. Application Layer –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3KCnWXq    #systemdesign #coding #interviewtips  .

Why is Nginx called a “𝐫𝐞𝐯𝐞𝐫𝐬𝐞” proxy? . . The diagram below shows the differences between a 𝐟𝐨𝐫𝐰𝐚𝐫𝐝 𝐩𝐫𝐨𝐱𝐲 and a 𝐫𝐞𝐯𝐞𝐫𝐬𝐞 𝐩𝐫𝐨𝐱𝐲. 🔹 A forward proxy is a server that sits between user devices and the internet. A forward proxy is commonly used for:  1️⃣ Protect clients 2️⃣ Avoid browsing restrictions 3️⃣ Block access to certain content 🔹 A reverse proxy is a server that accepts a request from the client, forwards the request to web servers, and returns the results to the client as if the proxy server had processed the request. A reverse proxy is good for: 1️⃣ Protect servers 2️⃣ Load balancing 3️⃣ Cache static contents 4️⃣ Encrypt and decrypt SSL communications Over to you: What’s the difference between a reverse proxy and a load balancer? What are some of the most popular proxy servers? –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3KCnWXq    #systemdesign #coding #interviewtips  .

RPC vs. gRPC A remote procedure call (RPC) enables one machine to invoke some code on another machine as if it is a local function call from a user’s perspective. gRPC is an open-source remote procedure call framework created by Google in 2016. What makes gRPC so popular? - First, gRPC has a thriving developer ecosystem. The core of this ecosystem is the use of Protocol Buffers as its data interchange format. - The second reason why gRPC is so popular is that it is high-performance out of the box. –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3KCnWXq    #systemdesign #coding #interviewtips  .

Netflix Tech Stack (CI/CD Pipeline)    Planing: Netflix Engineering uses JIRA for planning and Confluence for documentation.    Coding: Java is the primary programming language for the backend service, while other languages are used for different use cases.    Build: Gradle is mainly used for building, and Gradle plugins are built to support various use cases.    Packaging: Package and dependencies are packed into an Amazon Machine Image (AMI) for release.    Testing: Testing emphasizes the production culture's focus on building chaos tools.    Deployment: Netflix uses its self-built Spinnaker for canary rollout deployment.    Monitoring: The monitoring metrics are centralized in Atlas, and Kayenta is used to detect anomalies.    Incident report: Incidents are dispatched according to priority, and PagerDuty is used for incident handling. –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3KCnWXq    #systemdesign #coding #interviewtips  .

What happens under the hood when you buy a product using PayPal or a bank card? To understand this, we need to digest two concepts: 𝐜𝐥𝐞𝐚𝐫𝐢𝐧𝐠 & 𝐬𝐞𝐭𝐭𝐥𝐞𝐦𝐞𝐧𝐭. Clearing is a process that calculates who should pay whom with how much money; while settlement is a process where real money moves between reserves in the settlement bank. Let’s say Bob wants to buy an SDI book from Claire’s shop on Amazon. - Pay-in flow (Bob pays Amazon money): 1.1 Bob buys a book on Amazon using Paypal. 1.2 Amazon issues a money transfer request to Paypal. 1.3 Since the payment token of Bob’s debit card is stored in Paypal, Paypal can transfer money, on Bob’s behalf, to Amazon’s bank account in Bank A. 1.4 Both Bank A and Bank B send transaction statements to the clearing institution. It reduces the transactions that need to be settled. Let’s assume Bank A owns Bank B $100 and Bank B owns bank A $500 at the end of the day. When they settle, the net position is that Bank B pays Bank A $400. 1.5 & 1.6 The clearing institution sends clearing and settlement information to the settlement bank. Both Bank A and Bank B have pre-deposited funds in the settlement bank as money reserves, so actual money movement happens between two reserve accounts in the settlement bank. - Pay-out flow (Amazon pays the money to the seller: Claire): 2.1 Amazon informs the seller (Claire) that she will get paid soon. 2.2 Amazon issues a money transfer request from its own bank (Bank A) to the seller bank (bank C). Here both banks record the transactions, but no real money is moved. 2.3 Both Bank A and Bank C send transaction statements to the clearing institution. 2.4 & 2.5 The clearing institution sends clearing and settlement information to the settlement bank. Money is transferred from Bank A’s reserve to Bank C’s reserve. Notice that we have three layers: - Transaction layer: where the online purchases happen - Payment and clearing layer: where the payment instructions and transaction netting happen - Settlement layer: where the actual money movement happen The first two layers are called information flow, and the settlement layer is called fund flow. You can see the 𝐢𝐧𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧 𝐟𝐥𝐨𝐰 𝐚𝐧𝐝 𝐟𝐮𝐧𝐝 𝐟𝐥𝐨𝐰 𝐚𝐫𝐞 𝐬𝐞𝐩𝐚𝐫𝐚𝐭𝐞𝐝. In the info flow, the money seems to be deducted from one bank account and added to another bank account, but the actual money movement happens in the settlement bank at the end of the day. Because of the asynchronous nature of the info flow and the fund flow, reconciliation is very important for data consistency in the systems along with the flow. It makes things even more interesting when Bob wants to buy a book in the Indian market, where Bob pays USD but the seller can only receive INR. –  Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://bit.ly/3KCnWXq    #systemdesign #coding #interviewtips  .