Logit and Sigmoid Functions and its Calculus
Definitions
The sigmoid function is defined as:
sigmoid(x) = 1 ÷ (1 + e^(-x))
The logit function is defined as:
logit(p) = ln (p ÷ (1 - p))
For logit(p) to have a real number answer, 0 ≤ p < 1
Transform from the Sigmoid Function to the Logit Function
We can easily transform from the sigmoid function to the logit function.
Let s = sigmoid(x). Then:
s = 1 ÷ (1 + e^(-x))
s * (1 + e^(-x)) = 1
s + s * e^(-x) = 1
s * e^(-x) = 1 - s
e^(-x) = (1 - s) ÷ s
e^x = s ÷ (1 - s)
x = ln(s ÷ (1 - s)) = logit(s)
To transform from the logit function to the sigmoid function, just go backwards.
Sigmoid Function: Derivative and Integral
Derivative
s = sigmoid(x)
s = 1 ÷ (1 + e^(-x))
Using the quotient rule of derivatives:
ds/dx = [(1 + e^(-x)) * 0 - 1 * -e^(-x)] ÷ (1 + e^(-x))^2
= -(-e^(-x)) ÷ (1 + e^(-x))^2
= -e^(-x) ÷ (1 + e^(-x))^2
Integral
s = sigmoid(x)
s = 1 ÷ (1 + e^(-x))
Multiply both sides by e^x ÷ e^x:
s * (e^x ÷ e^x) = (e^x ÷ e^x) * (1 ÷ (1 + e^(-x)))
s = e^x ÷ (e^x + 1)
Integral:
∫ e^x ÷ (e^x + 1) dx
Let u = e^x + 1. Then du = e^x dx
= ∫ du ÷ (u + 1)
= ln (u + 1) + C
= ln (e^x + 1) + C
Summary:
d/dx sigmoid(x) = -e^(-x) ÷ (1 + e^(-x))^2
∫ sigmoid(x) dx = ln (e^x + 1) + C
Logit Function: Derivative and Integral
Derivative
logit(p) = ln (p ÷ (1 - p))
L = ln (p ÷ (1 - p))
Derivative:
dL/dp = [(1 - p) ÷ p] * d/dp ln (p ÷ (1 - p))
= [(1 - p) ÷ p] * [(1 - p) * 1 - p * (-1)] ÷ [(1 - p)^2]
= [(1 - p) ÷ p] * [1 - p + p] ÷ [(1 - p)^2]
= [(1 - p) ÷ p] * 1 ÷ (1 - p)^2
= 1 ÷ [p * (1 - p)]
Integral:
∫ ln (p ÷ (1 - p)) dp
By integration by parts:
u = ln (p ÷ (1 - p))
du = 1 ÷ [p * (1 - p)] dp
v = dp
v = p
Then:
∫u dv
= p * ln ( p ÷ (1 - p)) - ∫ p ÷ (1 - p) dp
= p * ln ( p ÷ (1 - p)) + ∫ -p ÷ (1 - p) dp
= p * ln ( p ÷ (1 - p)) + ln(1 - p) + C
In Summary:
d/dp logit(p) = 1 ÷ [p * (1 - p)]
∫ logit(p) dp = p * ln ( p ÷ (1 - p)) + ln(1 - p) + C
Eddie
All original content copyright, © 2011-2022. Edward Shore. Unauthorized use and/or unauthorized distribution for commercial purposes without express and written permission from the author is strictly prohibited. This blog entry may be distributed for noncommercial purposes, provided that full credit is given to the author.
