## Saturday, September 17, 2022

### Logit and Sigmoid Functions and its Calculus

Logit and Sigmoid Functions and its Calculus

Definitions

The sigmoid function is defined as:

sigmoid(x) = 1 ÷ (1 + e^(-x))

The logit function is defined as:

logit(p) = ln (p ÷ (1 - p))

For logit(p) to have a real number answer, 0 ≤ p < 1

Transform from the Sigmoid Function to the Logit Function

We can easily transform from the sigmoid function to the logit function.

Let s = sigmoid(x). Then:

s = 1 ÷ (1 + e^(-x))

s * (1 + e^(-x)) = 1

s + s * e^(-x) = 1

s * e^(-x) = 1 - s

e^(-x) = (1 - s) ÷ s

e^x = s ÷ (1 - s)

x = ln(s ÷ (1 - s)) = logit(s)

To transform from the logit function to the sigmoid function, just go backwards.

Sigmoid Function:  Derivative and Integral

Derivative

s = sigmoid(x)

s = 1 ÷ (1 + e^(-x))

Using the quotient rule of derivatives:

ds/dx = [(1 + e^(-x)) * 0 - 1 * -e^(-x)] ÷ (1 + e^(-x))^2

= -(-e^(-x)) ÷ (1 + e^(-x))^2

= -e^(-x) ÷ (1 + e^(-x))^2

Integral

s = sigmoid(x)

s = 1 ÷ (1 + e^(-x))

Multiply both sides by e^x ÷ e^x:

s * (e^x ÷ e^x) = (e^x ÷ e^x) * (1 ÷ (1 + e^(-x)))

s = e^x ÷ (e^x + 1)

Integral:

∫ e^x ÷ (e^x + 1) dx

Let u = e^x + 1.  Then du = e^x dx

= ∫  du ÷ (u + 1)

= ln (u + 1) + C

= ln (e^x + 1) + C

Summary:

d/dx sigmoid(x) = -e^(-x) ÷ (1 + e^(-x))^2

∫ sigmoid(x) dx = ln (e^x + 1) + C

Logit Function:  Derivative and Integral

Derivative

logit(p) = ln (p ÷ (1 - p))

L = ln (p ÷ (1 - p))

Derivative:

dL/dp =  [(1 - p) ÷ p] * d/dp ln (p ÷ (1 - p))

=  [(1 - p) ÷ p] * [(1 - p) * 1 - p * (-1)] ÷ [(1 - p)^2]

=  [(1 - p) ÷ p] * [1 - p + p] ÷ [(1 - p)^2]

=  [(1 - p) ÷ p] * 1 ÷ (1 - p)^2

= 1 ÷ [p * (1 - p)]

Integral:

∫ ln (p ÷ (1 - p)) dp

By integration by parts:

u = ln (p ÷ (1 - p))

du = 1 ÷ [p * (1 - p)] dp

v = dp

v = p

Then:

∫u dv

= p * ln ( p ÷ (1 - p)) - ∫ p ÷ (1 - p) dp

= p * ln ( p ÷ (1 - p)) + ∫ -p ÷ (1 - p) dp

= p * ln ( p ÷ (1 - p)) + ln(1 - p) + C

In Summary:

d/dp logit(p) = 1 ÷ [p * (1 - p)]

∫ logit(p) dp = p * ln ( p ÷ (1 - p)) + ln(1 - p) + C

Eddie

All original content copyright, © 2011-2022.  Edward Shore.   Unauthorized use and/or unauthorized distribution for commercial purposes without express and written permission from the author is strictly prohibited.  This blog entry may be distributed for noncommercial purposes, provided that full credit is given to the author.

### Matrices in Python without Numpy: Part 1

Matrices in Python without Numpy:  Part 1 Introduction Python is a wonderful programming language and is a welcome addition to graphing calc...