SVM for GATE DA: Margins, Kernels & Worked Examples
Everything the exam asks about support vector machines — the 2/‖w‖ margin formula worked on numbers, support vectors identified by hand, soft margins and C, and the kernel trick computed step by step.
By Piyush Wairale · GATE DA Educator & Course Instructor, IIT Madras BS Programme · Updated August 2026
Key Takeaways
• An SVM picks the separating hyperplane w·x + b = 0 with the largest margin; the margin width is 2/‖w‖, so maximising margin = minimising ‖w‖².
• Only the support vectors — points on the margin boundaries where y(w·x+b) = 1 — determine the solution. Delete any other point and nothing changes.
• Soft margin adds slack ξᵢ with penalty C: large C → narrow margin, few violations (overfit risk); small C → wide margin, more violations (underfit risk).
• The kernel trick computes inner products in a high-dimensional feature space without visiting it — K(x, z) = (x·z + 1)² or the RBF kernel — making circles and XOR separable.
On this page
Maximum-margin intuition · The margin math (worked) · Soft margin, slack & C · Hinge loss · The kernel trick (worked) · SVM vs logistic regression · GATE-style solved problems · Common mistakes · Exam patterns · FAQs
Maximum Margin: Why “Any Separating Line” Isn’t Good Enough
If two classes are linearly separable, infinitely many hyperplanes separate them. A line that skims past training points classifies them correctly but generalises badly — a tiny perturbation flips predictions. The SVM answer: choose the hyperplane that stays as far as possible from both classes. That distance buffer is the margin, and the classifier that maximises it is provably the most robust linear separator. This one idea — margin maximisation — generates every formula below, and it’s listed explicitly in the official GATE DA syllabus under supervised learning.
For where SVM sits among the other classifiers you must know, see the Machine Learning pillar guide.
▶ Watch: my complete Machine Learning playlist for GATE DA
SVM lectures with worked numericals included — see all subject-wise playlists →
The Margin Math, Worked on Numbers
The decision boundary is w·x + b = 0. Scale w, b so the closest points of each class satisfy w·x + b = ±1 — these two parallel lines are the margin boundaries. The perpendicular distance between them is:
Maximising 2/‖w‖ is the same as minimising ½‖w‖² subject to yᵢ(w·xᵢ + b) ≥ 1 for all i — a convex quadratic programme with a unique optimum.
Worked (a): margin from w
An SVM returns w = (3, 4), b = −12. Then ‖w‖ = √(9+16) = 5, so the margin is 2/5 = 0.4. And the point (1, 1) sits at distance |3+4−12|/5 = 5/5 = 1 from the boundary. Both computations are staple 1-mark NATs.
Worked (b): identify the support vectors
Dataset: class +1 at (2, 0) and (3, 1); class −1 at (0, 0) and (−1, 1). The optimal boundary is the vertical line x₁ = 1, i.e. w = (1, 0), b = −1. Check the constraint value yᵢ(w·xᵢ+b) for each point: (2,0) → 1(2−1) = 1; (3,1) → 1(3−1) = 2; (0,0) → −1(0−1) = 1; (−1,1) → −1(−1−1) = 2.
Points achieving exactly 1 are the support vectors: (2, 0) and (0, 0). Margin = 2/‖(1,0)‖ = 2. The other two points could be deleted without moving the boundary — that “which points matter?” question is a recurring GATE MCQ.
Soft Margin: Slack Variables and the C Parameter
Real data is rarely perfectly separable — one mislabelled point would make the hard-margin problem infeasible. The soft-margin SVM allows violations by giving each point a slack ξᵢ ≥ 0 and paying for it in the objective:
Interpret ξᵢ: 0 means the point is safely outside the margin; 0 < ξᵢ ≤ 1 means inside the margin but correctly classified; ξᵢ > 1 means misclassified. C controls the trade-off — and its direction is the most-tested SVM fact in GATE:
Large C: violations are expensive → narrow margin, fits training data tightly, low bias / high variance — overfitting risk. Small C: violations are cheap → wide margin, tolerates errors, high bias / low variance — underfitting risk. (Careful: this is the opposite direction to λ in ridge regression — C multiplies the error term, λ multiplies the penalty term. Effectively C ≈ 1/λ.)
Hinge Loss: the SVM’s Objective in Loss Form
The soft-margin problem is equivalent to minimising regularised hinge loss:
Read it off: a point with y·s ≥ 1 (outside the margin) contributes zero loss; loss grows linearly as the point slides inside the margin or across the boundary. Quick evaluations GATE has asked: y = +1, s = 0.3 → loss = 0.7; y = −1, s = 0.5 → loss = 1.5; y = +1, s = 2 → loss = 0. Contrast with log-loss, which is never exactly zero — logistic regression always feels every point, the SVM only feels margin violators and support vectors.
The Kernel Trick, Computed by Hand
A linear boundary cannot separate concentric circles or XOR. Fix: map inputs to a higher-dimensional feature space φ(x) where they are linearly separable. The expensive part would be computing φ(x) — but the SVM’s optimisation only ever needs inner products φ(x)·φ(z). A kernel computes that product directly in the input space:
Polynomial: K(x, z) = (x·z + 1)d. RBF (Gaussian): K(x, z) = exp(−‖x−z‖²/2σ²), an infinite-dimensional feature space.
Worked (c): a kernel computation
Let x = (1, 2), z = (3, 1) and K(x, z) = (x·z + 1)². Then x·z = 3 + 2 = 5, so K = (5+1)² = 36.
Why is this a “trick”? For d = 2 the implicit map is φ(x) = (x₁², x₂², √2x₁x₂, √2x₁, √2x₂, 1) — six dimensions. Computing φ(x)·φ(z) explicitly: (1)(9) + (4)(1) + 2(2)(3) + 2(1)(3) + 2(2)(1) + 1 = 9 + 4 + 12 + 6 + 4 + 1 = 36. Same answer, but the kernel needed one dot product and one squaring. That equality — verified on numbers — is exactly what “the kernel trick” means, and GATE has asked it verbatim.
SVM vs Logistic Regression: the Comparison GATE Sets Up
| Aspect | SVM | Logistic regression |
|---|---|---|
| Loss | Hinge: max(0, 1−ys) | Log-loss (cross-entropy) |
| Output | Class / signed score | Calibrated probability |
| Solution depends on | Support vectors only | Every training point |
| Nonlinearity | Kernel trick (natural fit) | Manual feature engineering |
| Regularisation knob | C (large C → less regularised) | λ (large λ → more regularised) |
One-line summary worth memorising: need probabilities → logistic; need maximum-margin robustness or kernels → SVM.
Three GATE-Style Problems, Solved
Problem 1 (NAT). In one dimension, the support vectors are x = 3 (class +1) and x = 1 (class −1). Find w, b and the margin.
Solution. Support vectors satisfy the margin equations exactly: 3w + b = +1 and w + b = −1. Subtracting: 2w = 2 ⇒ w = 1, then b = −2 (boundary at x = 2). Margin = 2/|w| = 2 — which checks out: the gap between x = 1 and x = 3 is exactly 2.
Problem 2 (NAT). Scores on three points: (y = +1, s = 1.5), (y = +1, s = 0.2), (y = −1, s = −0.3). Total hinge loss?
Solution. Point 1: ys = 1.5 ≥ 1 → 0. Point 2: ys = 0.2 → 1 − 0.2 = 0.8. Point 3: ys = 0.3 → 1 − 0.3 = 0.7. Total = 0 + 0.8 + 0.7 = 1.5. Note point 3 is correctly classified (ys > 0) yet still penalised — it sits inside the margin. That subtlety is the intended trap.
Problem 3 (NAT). RBF kernel K(x, z) = exp(−‖x−z‖²/2σ²) with x = (1, 1), z = (4, 5), σ² = 12.5. Compute K.
Solution. ‖x−z‖² = 3² + 4² = 25. Denominator 2σ² = 25. K = e−1 ≈ 0.368. Sanity checks worth knowing: K = 1 when x = z, and K → 0 as points move far apart — the RBF kernel behaves like a similarity score.
Common Mistakes to Avoid
Margin = 1/‖w‖. The full margin (boundary-line to boundary-line) is 2/‖w‖; the distance from the hyperplane to one margin line is 1/‖w‖. Read which one the question asks.
Reversing C. Large C means less tolerance for violations, not more. Map C to 1/λ and reuse your ridge intuition.
Calling every training point a support vector. Only points with y(w·x+b) = 1 (or margin violators in the soft case) carry nonzero dual weight.
Thinking hinge loss = 0 means correct classification. It means correct and outside the margin. Correct-but-inside still costs 1 − ys.
Expanding kernels wrongly. (x·z + 1)² is a scalar squared — compute the dot product first, then square. Students who expand component-wise first routinely drop the cross terms.
How GATE DA Asks SVM
Four patterns dominate: (1) NAT — margin or point-to-hyperplane distance from a given w, b (needs the norm, so brush up via the Linear Algebra pillar); (2) NAT — hinge-loss or kernel-value computation; (3) MCQ — support-vector identification, effect of C, or removing a non-support point; (4) MCQ — SVM vs logistic properties. Expect 1–2 marks most years, occasionally linked with the regression material covered in the regression comparison guide.
Master the full ML syllabus for GATE DA 2027
My complete Machine Learning course covers SVM, regression, trees, naive Bayes, clustering, neural networks and PCA with recorded lectures, notes and GATE-level practice — aligned exactly to the DA syllabus.
Explore the Machine Learning Course →FAQs: SVM for GATE DA
Is SVM explicitly in the GATE DA syllabus?
Yes — support vector machines are named in the Machine Learning section of the official syllabus, alongside regression, trees, naive Bayes and neural networks.
Do I need the dual formulation and KKT conditions?
Deep derivations haven’t been asked. Know that the dual depends only on inner products (which is why kernels work) and that support vectors have nonzero dual multipliers. That level is sufficient.
What happens to the SVM if I delete a non-support point?
Nothing — the solution is unchanged. Deleting a support vector, however, can move the boundary. This asymmetry is a classic MCQ.
When should I choose RBF over a polynomial kernel?
RBF is the default for unknown, smooth nonlinear structure and behaves like a local similarity measure; polynomial kernels suit interactions of a known bounded degree. For GATE, focus on computing both on small numbers.
What should I study next?
Decision trees and naive Bayes — the next classifiers in the syllabus. The ML pillar guide lays out the full order.
Keep going: revise the complete ML roadmap, compare SVM against the linear models in the regression comparison guide, sharpen norms and dot products with the Linear Algebra pillar, and track coverage against the GATE DA 2027 syllabus. New ML problem-solving sessions drop regularly on my YouTube channel — subscribe so you don’t miss them.
Recent Post
A* search and alpha-beta pruning for GATE DA 2027: full open/closed-list trace, admissible heuristics, minimax with pruning counted leaf by leaf, solved problems.
Taylor series and maxima-minima for GATE DA 2027: standard expansions, e^0.1 and cos(0.2) approximated, derivative tests and the Hessian rule worked with solved problems.
Normal forms for GATE DA 2027: functional dependencies, attribute closure worked, 1NF to BCNF with full decompositions, checklist table and solved GATE problems.
SQL and relational algebra for GATE DA 2027: σ, π and joins worked on sample tables, GROUP BY and nested queries evaluated row by row, plus solved GATE problems.
BFS and DFS for GATE DA 2027: graph traversals traced step by step with queue and stack states, shortest paths, Python code, complexity and solved GATE problems.
Sorting algorithms in Python for GATE DA 2027: bubble, insertion, selection, merge and quick sort traced step by step, binary search, complexity table and solved problems.
Learn Daily, Wherever You Are
Free lectures, exam updates, PYQ discussions, and job alerts — delivered through our YouTube channel and Telegram communities.
