GATE 2025 DA Q.48: solution and answer

GATE 2025 · DAQ.48 · 2 marks · MSQ

Which of the following statements is/are correct about the rectified linear unit (ReLU) activation function defined as $ReLU (x) = max (x, 0)$ , where $x \in R$ ?

(A)
ReLU is continuous everywhere
(B)
ReLU is differentiable everywhere
(C)
ReLU is not differentiable at $x = 0$
(D)
$ReLU (x) = ReLU (a x)$ , for all $a \in R$

See the question in the paper

Tap to open full size

Answer and solution

ANSWER

The correct statements are (A) and (C): ReLU is continuous everywhere but not differentiable at $x = 0$ .

Official answer: A, CSource: Official answer key, GATE 2025 DA (IIT Roorkee)

Write ReLU piecewise: $ReLU (x) = 0$ for $x \leq 0$ and $ReLU (x) = x$ for $x > 0$ .
Continuity: both pieces are linear, so the only point to check is $x = 0$ . The left limit is $0$ , the right limit is $0$ , and $ReLU (0) = 0$ . So ReLU is continuous everywhere, and (A) is correct.
Differentiability at $x = 0$ : the left derivative is $lim_{h \to 0^{-}} \frac{0 - 0}{h} = 0$ . The right derivative is $lim_{h \to 0^{+}} \frac{h - 0}{h} = 1$ . They differ, so the derivative does not exist at $0$ (a corner).
Hence (B) is false, since ReLU is not differentiable at $0$ , and (C) is true.
Test (D) with a negative scale: take $a = - 1$ and $x = 1$ . Then $ReLU (1) = 1$ but $ReLU (- 1) = 0$ , so the equality fails and (D) is false. It holds only for $a \geq 0$ (positive homogeneity), not for every real $a$ .
Correct options: (A) and (C).

KEY IDEA

ReLU is continuous everywhere, but it has a corner at $0$ where the left slope is $0$ and the right slope is $1$ , so it is not differentiable there. It is positively homogeneous, so $ReLU (a x) = a ReLU (x)$ only for $a \geq 0$ .

WATCH OUT FOR

Treating ReLU as differentiable everywhere because it is piecewise linear, or assuming $ReLU (a x) = ReLU (x)$ for every real $a$ . Negative $a$ flips the sign of the input and changes the output.

From the GATE 2025 DA paper. Every question of this paper · Download the full paper · All 5 Neural networks questions, with notes

GATE 2025 DA Q.48: solution and answer

Answer and solution

More Neural networks questions