By Piyush Wairale · GATE DA Educator & Course Instructor, IIT Madras BS Programme · Updated August 2026

Why Matrix Decompositions Matter in GATE DA

The official GATE DA syllabus lists LU decomposition and singular value decomposition (SVD) by name under Linear Algebra. That makes them unusual: most syllabus lines are broad topics, but these are two specific algorithms the paper-setters can test procedurally. If you can run each factorisation by hand on a small matrix, the questions become free marks.

They also matter beyond the Linear Algebra section. LU is how numerical libraries actually solve linear systems (NumPy’s solve uses it), and SVD powers PCA, low-rank compression and recommender systems — all of which sit in the Machine Learning part of the syllabus. Learn the factorisations once and you reuse them across the paper. New to the section? Start with the full Linear Algebra for GATE DA pillar guide, then come back here for the deep dive.

LU Decomposition Step by Step (Doolittle Method)

LU decomposition writes a square matrix as A = LU, where L is lower-triangular with 1s on the diagonal (Doolittle convention) and U is upper-triangular. The insight to internalise: LU is just Gaussian elimination with the multipliers remembered. Every time you eliminate an entry using multiplier m, that m goes into L at the same position.

Worked example: factorise a 3×3 matrix

Factorise A = [ [2, 1, 1], [4, −6, 0], [−2, 7, 2] ].

Step 1 — eliminate column 1. Pivot = 2. Multipliers: m₂₁ = 4/2 = 2, m₃₁ = −2/2 = −1.

R2 ← R2 − 2R1 = [4−4, −6−2, 0−2] = [0, −8, −2]. R3 ← R3 − (−1)R1 = [−2+2, 7+1, 2+1] = [0, 8, 3].

Step 2 — eliminate column 2. Pivot = −8. Multiplier: m₃₂ = 8/(−8) = −1. R3 ← R3 − (−1)R2 = [0, 8−8, 3−2] = [0, 0, 1].

Step 3 — read off the answer. U keeps the reduced rows; L keeps the multipliers below a unit diagonal:

Verify: multiply LU and you recover A exactly. Bonus GATE shortcut: det(A) = det(L)·det(U) = 1 × (2)(−8)(1) = −16 — the product of U’s pivots. This one-liner alone has earned marks in previous GATE papers.

Two cautions. First, if a pivot turns out to be 0 you must swap rows first — then the factorisation is PA = LU with a permutation matrix P. Second, when A is invertible and no row swaps are needed, the Doolittle LU factorisation is unique.

Solving Ax = b with LU: Forward and Back Substitution

Why factorise at all? Because once A = LU, solving Ax = b never needs elimination again — just two triangular solves. Using the L and U from above, solve Ax = b for b = [5, −2, 9].

Stage 1 — forward substitution, Ly = b (top to bottom):

y₁ = 5.  Then 2y₁ + y₂ = −2 ⇒ y₂ = −2 − 10 = −12.  Then −y₁ − y₂ + y₃ = 9 ⇒ −5 + 12 + y₃ = 9 ⇒ y₃ = 2.

Stage 2 — back substitution, Ux = y (bottom to top):

Row 3: x₃ = 2.  Row 2: −8x₂ − 2(2) = −12 ⇒ x₂ = 1.  Row 1: 2x₁ + 1 + 2 = 5 ⇒ x₁ = 1.

So x = (1, 1, 2). Substitute into the original A to confirm: 2+1+2 = 5 ✓, 4−6+0 = −2 ✓, −2+7+4 = 9 ✓. For k different right-hand sides you factorise once and repeat only the cheap substitutions — this is exactly why solvers prefer LU over recomputing elimination.

SVD: Geometry First, Then a Full 2×2 Worked Example

The singular value decomposition writes A = UΣV^T: U and V are orthogonal (rotations/reflections), Σ is diagonal with non-negative entries σ₁ ≥ σ₂ ≥ … ≥ 0. Geometrically, every matrix — square or rectangular — acts on space as: rotate (V^T), stretch along axes (Σ), rotate again (U).

Worked example: full SVD of a 2×2 matrix

Compute the SVD of A = [ [3, 1], [1, 3] ].

Step 1 — form A^TA. A^TA = [ [10, 6], [6, 10] ].

Step 2 — eigenvalues of A^TA. det(A^TA − λI) = (10−λ)² − 36 = 0 ⇒ 10−λ = ±6 ⇒ λ₁ = 16, λ₂ = 4.

Step 3 — singular values. σ₁ = √16 = 4, σ₂ = √4 = 2. So Σ = diag(4, 2).

Step 4 — right singular vectors (V). For λ = 16: (10−16)v₁ + 6v₂ = 0 ⇒ v₁ = v₂ ⇒ v₁ = (1/√2)[1, 1]. For λ = 4: v₂ = (1/√2)[1, −1].

Step 5 — left singular vectors (U) via uᵢ = Avᵢ/σᵢ: Av₁ = (1/√2)[4, 4], divide by 4 ⇒ u₁ = (1/√2)[1, 1]. Av₂ = (1/√2)[2, −2], divide by 2 ⇒ u₂ = (1/√2)[1, −1].

Result: A = UΣV^T with U = V = (1/√2)[ [1, 1], [1, −1] ] and Σ = diag(4, 2). (U = V here because A is symmetric positive definite — a pattern worth remembering for the exam.)

Rank-1 Approximation and the Image-Compression Intuition

Expand the SVD as a sum of rank-1 pieces: A = σ₁u₁v₁^T + σ₂u₂v₂^T + … Because the σ’s are sorted, truncating after k terms gives the best rank-k approximation of A (Eckart–Young theorem), with spectral-norm error exactly σ_k+1.

For our worked matrix: A₁ = σ₁u₁v₁^T = 4 · ½ [ [1, 1], [1, 1] ] = [ [2, 2], [2, 2] ], and the error A − A₁ = [ [1, −1], [−1, 1] ] has norm σ₂ = 2, exactly as the theorem promises.

Why data scientists care: a 1000×1000 image is a million numbers, but if its singular values decay fast, keeping just 50 triplets (σᵢ, uᵢ, vᵢ) ≈ 100k numbers reconstructs it almost perfectly — that is SVD image compression. The same truncation is what PCA does to data covariance in the ML syllabus: keep the directions with the largest σ, drop the noise.

Eigenvalues vs Singular Values: the Comparison GATE Tests

Three GATE-Style Problems, Solved

Problem 1 (1 mark). The singular values of A = [ [2, 0], [0, −3] ] are:

Solution. A^TA = diag(4, 9), eigenvalues 4 and 9, so σ = 3 and 2 (sorted descending). Answer: 3, 2 — not 2 and −3. Singular values are never negative; this sign trap is a favourite.

Problem 2 (2 marks). In the Doolittle factorisation of A = [ [4, 3], [6, 3] ], find l₂₁ and u₂₂, and hence det(A).

Solution. l₂₁ = 6/4 = 1.5. R2 ← R2 − 1.5R1 gives u₂₂ = 3 − 1.5(3) = −1.5. det(A) = u₁₁u₂₂ = 4(−1.5) = −6 (check: 4·3 − 3·6 = −6 ✓).

Problem 3 (2 marks). A 2×2 matrix has singular values 3 and 2. Find ‖A‖_F² and the spectral-norm error of its best rank-1 approximation.

Solution. ‖A‖_F² = σ₁² + σ₂² = 9 + 4 = 13. Best rank-1 error = σ₂ = 2 by Eckart–Young. No computation of A itself is ever needed — recognising that is the whole question.

Common Mistakes to Avoid

Sign errors in multipliers. L stores the multiplier you subtracted. If R3 ← R3 + R1, the multiplier was −1 and l₃₁ = −1, not +1.

Reporting negative singular values. σᵢ ≥ 0 always. For a symmetric matrix with eigenvalue −3, the corresponding singular value is 3.

Forgetting to sort. Convention puts σ₁ ≥ σ₂; answers like “2, 3” get marked wrong in NAT questions asking for σ₁.

Mixing up the substitution order. Forward substitution goes with L (top row first), back substitution with U (bottom row first). Swapping them is the fastest way to nonsense.

Assuming every matrix has an LU factorisation. A zero pivot forces a row swap (PA = LU). SVD, by contrast, never fails — it exists for every matrix.

How GATE DA Asks This Topic

Across GATE DA and neighbouring papers (CS, engineering maths), decomposition questions cluster into four patterns: (1) NAT: compute one entry of L or U, or a determinant via pivots; (2) NAT/MCQ: singular values of a small matrix, or ‖A‖_F from given σ’s; (3) MCQ: properties — existence, uniqueness, U/V orthogonality, eigen-vs-singular relations; (4) linked context: SVD inside a PCA or low-rank compression question. Budget 2–3 minutes each; the arithmetic is deliberately small. See the complete GATE DA 2027 syllabus breakdown for where Linear Algebra sits in the paper.

FAQs: SVD and LU for GATE DA

Keep the momentum going: work through the full Linear Algebra roadmap, see how SVD powers PCA in the Machine Learning guide, and check the GATE DA 2027 syllabus to plan your next topic. New lectures drop regularly on my YouTube channel — subscribe so you don’t miss the Linear Algebra problem-solving sessions.