DS Neural Networks

A neural network is a model made of small calculating units called neurons. The units connect in layers, and the connections carry adjustable numbers called weights. Training adjusts the weights until the network predicts well. Neural networks power image recognition, voice assistants, and language tools.

Layman View: A Team of Voters

Imagine a team deciding whether to carry an umbrella. One member checks clouds, another checks the forecast, and a third checks the wind. Each member votes with a different strength. A manager adds the votes and decides. A neural network works like many such teams stacked in rows, and training teaches each member how much to trust each input.

The Structure

 INPUT LAYER     HIDDEN LAYER     OUTPUT LAYER

   (x1) ------\  /--- (h1) ---\
               \/             \
   (x2) -------/\--- (h2) ----->  (y)  --> prediction
               /\             /
   (x3) ------/  \--- (h3) ---/

 Every line carries a weight. Every circle adds its inputs,
 then passes the sum through an activation function.

Part	Role
Input layer	Receives the raw features
Hidden layers	Find patterns inside the data
Output layer	Gives the final prediction
Weights	Control how strongly one neuron influences the next
Bias	Shifts a neuron's decision point

Inside One Neuron

 output = activation( w1*x1 + w2*x2 + w3*x3 + bias )

The neuron multiplies each input by its weight, adds the results, and applies an activation function. Without activation functions, a network could learn only straight-line patterns.

Common Activation Functions

Name	Behavior	Typical Use
ReLU	Returns 0 for negatives and the same number for positives	Hidden layers
Sigmoid	Squeezes any number into 0 to 1	Yes or no output
Softmax	Turns scores into chances that add to 1	Multi-class output

How Training Works

  Feed data forward --> Compare with true answer --> Measure the error (loss)
          ^                                                    |
          |                                                    v
  Update weights  <---------  Send error backward (backpropagation)

Loss

The loss function scores how wrong a prediction is. Lower loss means better predictions.

Gradient descent

Picture walking downhill in fog. You feel the slope under your feet and step downward. Gradient descent adjusts every weight a small step in the direction that lowers the loss. The learning rate sets the size of each step. Steps that are too large overshoot the valley, and steps that are too small waste time.

Epochs and batches

An epoch is one full pass through the training data. Data arrives in small batches, and the weights update after each batch.

Build a Network with Keras

from tensorflow import keras
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X, y = make_classification(n_samples=1000, n_features=30,
                           n_informative=10, random_state=1)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=1)

scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

model = keras.Sequential([
    keras.layers.Dense(16, activation="relu", input_shape=(30,)),
    keras.layers.Dense(8, activation="relu"),
    keras.layers.Dense(1, activation="sigmoid")
])

model.compile(optimizer="adam", loss="binary_crossentropy",
              metrics=["accuracy"])
model.fit(X_train, y_train, epochs=30, batch_size=16,
          validation_split=0.2)
print(model.evaluate(X_test, y_test))

The sample data holds 1000 rows with 30 numeric columns and two classes, and the network learns to separate them. The validation split watches progress on data the network does not train on.

Fight Overfitting

• Add dropout layers that switch off random neurons during training.
• Stop training early when validation loss stops improving.
• Collect more data or augment existing data.
• Reduce the number of layers or neurons.

Popular Network Types

Type	Best For
Dense network	Tables of numbers
Convolutional network	Images
Recurrent network	Sequences such as text and sound
Transformer	Language and large-scale sequence tasks

When to Choose Neural Networks

Neural networks shine with large datasets and unstructured data such as images, audio, and text. Tables with a few thousand rows often work better with random forests or regression, and those models train faster and explain themselves more easily.

Key Points

• Layers of neurons transform inputs into predictions through weights and activations.
• Training repeats forward passes, loss checks, and weight updates.
• Regularization methods keep networks from memorizing the training data.