Synopsis
Free style Software engineering talk.
Episodes
-
The Cloudflare mTLS vulnerability - A Deep Dive Analysis
06/04/2023 Duration: 43minCloudflare released a blog detailing a vulnerability that has been in their system for nearly two years. it is related to mTLS or mutual TLS and specifically client certificate revocation. I explore this in details 0:00 Intro 3:00 The Vulnerability 7:00 What happened? 8:50 Certificate Revocation 12:30 Rejecting certain endpoints 17:00 Certificate Authentication 20:30 Certificate serial number 24:00 Session Resumption (PSK) 35:00 The bug 37:00 How they addressed the problem Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
-
The Virgin Media ISP outage - What happened?
06/04/2023 Duration: 23minBGP (Border gateway protocol) withdrawals caused the Virgin media ISP customers to lose their Internet connection. I go into details on this video. 0:00 Intro 2:00 What happened? 4:11 How BGP works? 11:50 Version media withdrawals 15:00 Deep dive Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
-
GitHub SSH key is Leaked - How bad is this?
30/03/2023 Duration: 21minGitHub Accidentally Exposed their SSH RSA Private key, this is the message you will get . @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the RSA key sent by the remote host is SHA256:uNiVztksCsDhcc0u9e8BujQXVUpKZIDTMczCvj3tD2s. Please contact your system administrator. Add correct host key in ~/.ssh/known_hosts to get rid of this message. Host key for github.com has changed and you have requested strict checking. Host key verification failed. In this video I discuss how bad is this,. 0:00 Intro 1:10 What happened? 3:00 SSH vs TLS Authentication 6:00 SSH Connect 7:45 How bad is the github leak? 15:00 What should you do? 18:50 Is ECDSA immune? https://github.blog/2023-03-23-
-
Cookie Hijacking - How Linus Tech Tips got Hacked
29/03/2023 Duration: 13minHow Linus Tech Tips channel got Hacked In this short video we explain how was it possible for Linux to get hacked with cookies hijacking. 0:00 Intro 0:47 TLDR what happened 5:10 Cookies in Chrome 7:30 Cookies Hijacking 8:46 Session Tokens (Access/Refresh) 10:00 Remedies
-
All Postgres Locks Explained | A Deep Dive
19/03/2023 Duration: 49minGet my database engineering course https://database.husseinnasser.com In this video I do a deep dive in all locks obtained by postgres, I learned a lot while making this video and hope you enjoy it. 0:00 Intro 2:30 What are Locks? 5:30 Overview of Postgres Locks 9:10 Table-Level Locks 11:40 ACCESS EXCLUSIVE 17:40 ACCESS SHARE 19:00 ROW SHARE 20:15 ROW EXCLUSIVE 21:15 SHARE UPDATE EXCLUSIVE 23:30 SHARE 24:50 SHARE ROW EXCLUSIVE 25:18 EXCLUSIVE 25:30 Table Lock Conflict Matrix 28:30 Row-Level Locks 30:00 FOR UPDATE 33:00 FOR NO KEY UPDATE 34:00 FOR SHARE 34:40 FOR KEY SHARE 35:10 Row Lock Conflict Matrix 39:25 Page-Level Locks 42:00 Deadlocks 46:00 Advistory Locks 47:20 Summary https://www.postgresql.org/docs/current/explicit-locking.html
-
Pinterest moves to HTTP/3
16/03/2023 Duration: 25minPinterest moves to HTTP/3 on all their clients and edge CDNs this year. They witnessed interesting gains but not without good lesson learned. The main one was the mismatch of alt-svc vs DNS ttls. I cover this on the next episode of the backend engineering course. 0:00 Intro 2:00 Moving h2 to h3 through alt-svc 5:00 Why HTTP/3 6:00 HTTP/1 vs HTTP/2 9:00 TCP Head of Line blocking in HTTP/2 11:00 How HTTP/3 addresses HOL 12:15 Connection Migration 13:30 Stream level congestion control 14:10 1-RTT - 0-RTT 15:41 Pinterest challenges moving HTTP/3 19:00 Migration 21:15 Future work 22:30 Summary article https://medium.com/pinterest-engineering/pinterest-is-now-on-http-3-608fb5581094 Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
-
Why Loom Users got each others’ sessions on March 7th 2023
14/03/2023 Duration: 14minOn March 7 2023, Loom users started seeing each others data as a result of cookies getting leaked from the CDN. This loom security breach is really critical. Let us discuss 0:00 Intro 1:00 Why Cookies 2:00 How this happens 5:50 What caused it? 7:30 How Loom solved it? 8:20 Reading the RCA 10:30 Remedies
-
How Discord Stores Trillions of Messages - A deep dive
11/03/2023 Duration: 01h09minDiscord engineering goes into details of how they migrated from Cassandra to ScyllaDB, improved the performance of their reads and writes and rearchitected their backend to support the new load. It is an interesting episode lets get into it 0:00 Intro 1:50 Relational vs Distributed 7:00 The Cassandra Troubles 11:00 SnowFlake vs UUID 14:30 B+Tree 19:20 B+Tree and SSDs 25:30 LSM Trees 31:00 Hot partitions 36:00 Cassandra Garbage Collector Pauses 40:00 Changing the Architecture 45:00 The Data Services 55:00 The Migration 1:02:00 Zoned Named Spaces 1:04:00 Summary Article here How Discord Stores Trillions of Messages https://discord.com/blog/how-discord-stores-trillions-of-messages
-
Postgres Architecture | The Backend Engineering Show
16/02/2023 Duration: 34minCreating a listener on the backend application that accepts connections is simple. You listen on an address-port pair, connection attempts to that address and port will get added to an accept queue; The application accepts connections from the queue and start reading the data stream sent on the connection. However, what part of your application does the accepting and what part does the reading and what part does the execution? You can architect your application in many ways based on your use cases. I have a medium post just exploring the different options. In this video I explore the PostgreSQL process architecture in details. Please note that the information here is derived from both the Postgres doc and code. Discussions about scalability and performance are solely based on my opinions. 0:00 Intro 1:30 Overview 3:30 Postgres MVCC 5:30 Processes vs Threads 7:40 Postmaster Process 8:00 Backend Processes 13:30 Shared Buffers 14:52 Background Workers 17:18 Auxiliary Processes 17:45 Background Writer 22:30 Check
-
How Alt-Svc switches HTTP/2 clients to use HTTP/3 | The Backend Engineering Show
13/02/2023 Duration: 23minThe Alt-Svc header/frame is a capability that allows the server to adverse alternative services to the connected application available in protocols, ports or domains. It is available as a response header alt-svc and also as an HTTP/2 frame. Let us discuss this capability. 0:00 Intro 1:38 what is alt-svc? 5:30 uses of h3 in alt-svc 8:00 alt-svc header 10:00 Alt-svc header with 103 early hints 14:48 h2 altsvc frame 18:30 SVCB DNS record 21:20 Summary Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
-
Your DNS queries will eventually look like this (0x20 DNS encoding)
28/01/2023 Duration: 26minCorrection: Google is implementing the proposal originally submitted by researchers from Georgia institute of tech. I incorrectly said in the video that google is proposing this . Google is finally implementing a proposal from 2008 by researchers from Georgia institute of technology to make DNS cache poisoning . https://astrolavos.gatech.edu/articles/increased_dns_resistance.pdf https://datatracker.ietf.org/doc/html/draft-vixie-dnsext-dns0x20-00 0:00 Intro 2:00 How DNS Work 5:00 DNS Cache Poisoning 14:00 gOoGLe dot CoM 16:20 ASCII 0x20 casing 18:30 Randomizing the casing with encryption 22:30 limitations of this proposal 24:00 Credits
-
DropBox Removed their SSDs, got 20% faster writes
24/01/2023 Duration: 31minhttps://dropbox.tech/infrastructure/increasing-magic-pocket-write-throughput-by-removing-our-ssd-cache-disks In this episode of the backend engineering show I’ll discuss how Dropbox improved their write through put by 20% by removing all their SSDs (yes I was surprised too). DropBox uses an SSD layer as a write-back cache with SMR drives as their backend persistent storage. They changed their model to write directly to the hard drives. 0:00 Intro 2:00 Article Summary 3:00 SMR Drives 6:00 SSD Cache & WriteBack 8:00 Replacing Cache 9:30 Storage Engine Background 14:30 Why did they do it 15:00 The limitation of SSDs & Zoned Namespaces 19:30 Updating the Storage Engine 22:30 Tradeoffs 26:00 Rollout 28:00 Summary
-
MySQL on HTTP/3 | The Backend Engineering Show
05/01/2023 Duration: 37minThe communication between backend applications and database systems always fascinated me. The protocols keep evolving and we are in constant search for an efficient protocol that best fit the workload of Backend-DB communication. In this episode of the backend engineering show I go through a blog written by @PlanetScale doing an experimentation of using HTTP/3 and HTTP/2 comparing it with MySQL Binary protocol. https://planetscale.com/blog/faster-mysql-with-http3 0:00 Intro 7:45 MySQL Binary vs HTTP 10:20 The Tests 15:00 Connection Cost + Select 1 22:00 Parallel Select 26:00 The cost of H2 and H3
-
How Shopify’s engineering improved writes by 50% with ULID | The Backend Engineering Show
23/12/2022 Duration: 32minFundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com Shopify posted a blog on tips to for scalable payment system, one tip peeked my interest related to switching from UUID to ULID. I explore the reasoning behind this in this video. https://shopify.engineering/building-resilient-payment-systems 0:00 Intro 1:30 idempotency 6:30 UUID vs ULID 9:50 Clustered Index 13:30 Why UUID4 Inserts are slow 17:15 How ULID helps Shopify 22:00 Problem with tail pages 25:00 Does ULID help in all cases? Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
-
MongoDB Internal Architecture | The Backend Engineering Show
16/12/2022 Duration: 44minI’m a big believer that database systems share similar core fundamentals at their storage layer and understanding them allows one to compare different DBMS objectively. For example, How documents are stored in MongoDB is no different from how MySQL or PostgreSQL store rows. Everything goes to disk, the trick is to fetch what you need from disk efficiently with as fewer I/Os as possible, the rest is API. In this video I discuss the evolution of MongoDB internal architecture on how documents are stored and retrieved focusing on the index storage representation. I assume the reader is well versed with fundamentals of database engineering such as indexes, B+Trees, data files, WAL etc, you may pick up my database course to learn the skills. Let us get started. Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://netw
-
How UI/UX can break the backend
01/12/2022 Duration: 30minThe User Interface/User Experience has great impact on the backend architecture and scalability. In this podcast I discuss three UI/UX that affected backend design and scalability. 0:00 Intro 1:40 UI vs UX 4:30 Google Chrome OmniBox 12:30 1 out of X Page 20:00 YouTube Notification Resources https://blog.apnic.net/2020/08/21/chromiums-impact-on-root-dns-traffic/ Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
-
Do DHCP and DNS Servers Communicate?
16/11/2022 Duration: 30minIn this video I explain how DHCP work and how it updates DNS entries for new hosts joining the network. I'll also mention Zero Config 0:00 Intro 1:00 the Network configuration 6:00 Showing DHCP in Wireshark 6:30 DHCP Discover 14:40 DHCP Offer 19:00 DHCP Request 21:30 DHCP ACK 22:00 How DHCP Updates DNS 26:15 Zero Configuration (mDNS, Link-local) Resources Dhcp https://datatracker.ietf.org/doc/html/rfc1541 Dynamic updates , dhcp RFC2136 https://datatracker.ietf.org/doc/html/rfc2136 https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipaddr_dhcp/configuration/xe-3se/3850/dhcp-xe-3se-3850-book/dhcp-client-option-12.pdf RFC 1497 https://www.rfc-editor.org/rfc/rfc1497.html https://www.rfc-editor.org/rfc/rfc6762#ref-Zeroconf Link-local https://www.rfc-editor.org/rfc/rfc3927
-
Compressing Certificates in TLS | The Backend Engineering Show
08/11/2022 Duration: 34minFundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com Certificates provide a way to authenticate both the server and the client and are included as part of the TLS handshake. However, the certificates can be large because the full certificate chain is included in the handshake. The large certificates can go up to 10KB in size and take multiple segments to deliver and assemble. RFC 8879 discusses how TLS compression can be achieved, I discuss that in this podcast. Enjoy. 0:00 Intro 4:15 Certificate Chain 6:00 Faking the chain 8:50 Certificate Stores 10:30 Including ROOT cert in the chain 12:00 The performance penalty of large certificate chain 20:15 RFC 8879 TLS Certificate Compression 23:00 How Compression Works in TLS 1.2 vs TLS 1.3 30:30 What could go wrong? Resources https://datatracker.ietf.org/doc/rfc8879/ https://www.rfc-editor.org/rfc/rfc5246 https://www.rfc-editor.org/rfc/rfc6928.html
-
OpenSSL new vulnerability
06/11/2022 Duration: 10minTwo new vulnerabilities in openssl were discovered, we discuss them in this video https://www.openssl.org/news/secadv/20221101.txt
-
TCP Protective Load Balancing coming to Linux Kernel 6.2
03/11/2022 Duration: 14minGoogle recent paper on protective load balancing in TCP attempts to improve packet drops and latency by making the host change the flow path using the IPv6 Flow label. The Linux kernels gets the PLB support in Linux 6.2 this December, let us discuss with this is.