maint: Add config.hpp with default model hyperparameters and constants

README.md

@@ -14,6 +14,12 @@ Alongside it sits a parallel PyTorch implementation in [engine/main.py](engine/m
 
 The point of this repo is the C++ core. The PyTorch, FastAPI, and frontend layers exist to make the model usable, but if you're here to learn how a GPT is actually built and trained without a framework doing the work for you, [include/backward.h](include/backward.h) is where to start reading.
 
+<h1 align="center">
+<img width="824" height="250" alt="image" src="https://github.com/user-attachments/assets/5c65daa2-903a-4392-85e3-56442bb82cac" />
+
+
+</h1>
+
 ***technical notes***: [docs](https://eamon2009.github.io/LLMs/)
 
 

config/config.h

@@ -5,15 +5,15 @@ static const std::string DEFAULT_CLEANED_PATH = "data/input.txt";
 static const std::string DATA_PATH_ENV_VAR = "GPT_DATA_PATH";
 static const unsigned int SEED = 1337;
 static const double TRAIN_SPLIT = 0.9; // 90% train, 10% val
-static const int BATCH_SIZE = 16;      
-static const int BLOCK_SIZE = 64;       // Context length
-static const int MAX_ITERS = 50;     
-static const int EVAL_INTERVAL = 250; 
-static const float LEARNING_RATE = 5e-4f; 
-static const int EVAL_ITERS = 100;      
+static const int BATCH_SIZE = 16;
+static const int BLOCK_SIZE = 64; // Context length
+static const int MAX_ITERS = 50;
+static const int EVAL_INTERVAL = 250;
+static const float LEARNING_RATE = 1e-4f;
+static const int EVAL_ITERS = 100;
 static const int N_EMBD = 128;
 static const int N_HEAD = 4;
 static const int N_LAYER = 4;
-static const float DROPOUT = 0.05f; 
+static const float DROPOUT = 0.05f;
 static const std::string BEST_MODEL_PATH = "best_model.bin";
 static const std::string MODEL_PATH_ENV_VAR = "GPT_MODEL_PATH";