**Logit and Sigmoid Functions and its Calculus**

**Definitions**

The sigmoid function is defined as:

sigmoid(x) = 1 ÷ (1 + e^(-x))

The logit function is defined as:

logit(p) = ln (p ÷ (1 - p))

For logit(p) to have a real number answer, 0 ≤ p < 1

**Transform from the Sigmoid Function to the Logit Function**

We can easily transform from the sigmoid function to the logit function.

Let s = sigmoid(x). Then:

s = 1 ÷ (1 + e^(-x))

s * (1 + e^(-x)) = 1

s + s * e^(-x) = 1

s * e^(-x) = 1 - s

e^(-x) = (1 - s) ÷ s

e^x = s ÷ (1 - s)

x = ln(s ÷ (1 - s)) = logit(s)

To transform from the logit function to the sigmoid function, just go backwards.

**Sigmoid Function: Derivative and Integral**

Derivative

s = sigmoid(x)

s = 1 ÷ (1 + e^(-x))

Using the quotient rule of derivatives:

ds/dx = [(1 + e^(-x)) * 0 - 1 * -e^(-x)] ÷ (1 + e^(-x))^2

= -(-e^(-x)) ÷ (1 + e^(-x))^2

= -e^(-x) ÷ (1 + e^(-x))^2

Integral

s = sigmoid(x)

s = 1 ÷ (1 + e^(-x))

Multiply both sides by e^x ÷ e^x:

s * (e^x ÷ e^x) = (e^x ÷ e^x) * (1 ÷ (1 + e^(-x)))

s = e^x ÷ (e^x + 1)

Integral:

∫ e^x ÷ (e^x + 1) dx

Let u = e^x + 1. Then du = e^x dx

= ∫ du ÷ (u + 1)

= ln (u + 1) + C

= ln (e^x + 1) + C

Summary:

d/dx sigmoid(x) = -e^(-x) ÷ (1 + e^(-x))^2

∫ sigmoid(x) dx = ln (e^x + 1) + C

**Logit Function: Derivative and Integral**

Derivative

logit(p) = ln (p ÷ (1 - p))

L = ln (p ÷ (1 - p))

Derivative:

dL/dp = [(1 - p) ÷ p] * d/dp ln (p ÷ (1 - p))

= [(1 - p) ÷ p] * [(1 - p) * 1 - p * (-1)] ÷ [(1 - p)^2]

= [(1 - p) ÷ p] * [1 - p + p] ÷ [(1 - p)^2]

= [(1 - p) ÷ p] * 1 ÷ (1 - p)^2

= 1 ÷ [p * (1 - p)]

Integral:

∫ ln (p ÷ (1 - p)) dp

By integration by parts:

u = ln (p ÷ (1 - p))

du = 1 ÷ [p * (1 - p)] dp

v = dp

v = p

Then:

∫u dv

= p * ln ( p ÷ (1 - p)) - ∫ p ÷ (1 - p) dp

= p * ln ( p ÷ (1 - p)) + ∫ -p ÷ (1 - p) dp

= p * ln ( p ÷ (1 - p)) + ln(1 - p) + C

In Summary:

d/dp logit(p) = 1 ÷ [p * (1 - p)]

∫ logit(p) dp = p * ln ( p ÷ (1 - p)) + ln(1 - p) + C

Eddie

All original content copyright, © 2011-2022. Edward Shore. Unauthorized use and/or unauthorized distribution for commercial purposes without express and written permission from the author is strictly prohibited. This blog entry may be distributed for noncommercial purposes, provided that full credit is given to the author.