Update README.md

README.md

@@ -1,3 +1,51 @@
+# Model Details
+
+##### Model Name: NumericBERT
+
+##### Model Type: Transformer
+
+##### Architecture: BERT
+
+##### Training Method: Masked Language Modeling (MLM)
+
+##### Training Data: MIMIC IV Lab values data
+
+##### Training Hyperparameters:
+
+Optimizer: AdamW
+Learning Rate: 5e-5
+Masking Rate: 20%
+Tokenization
+Tokenizer: Custom numeric-to-text mapping using the TextEncoder class
+
+### Text Encoding Process:
+
+The process converts non-negative integers into uppercase letter-based representations. This mapping allows numerical values to be expressed as sequences of letters. 
+Subsequently, a method is applied to scale numerical values and convert them into corresponding letters based on a predefined mapping. 
+Finally, a text encoding is executed to add the corresponding lab ID using the numeric values in specified columns ('Bic', 'Crt', 'Pot', 'Sod', 'Ure', 'Hgb', 'Plt', 'Wbc').
+
+
+### Training Data Preprocessing
+Column Selection: Numerical values from the following lab values represented as: 'Bic', 'Crt', 'Pot', 'Sod', 'Ure', 'Hgb', 'Plt', 'Wbc'.
+Text Encoding: The numeric values are encoded into text.
+Masking: 20% of the data is randomly masked during training.
+
+### Model Output
+The model outputs predictions for masked values during training.
+The output contains the encoded text.
+
+### Limitations and Considerations
+Numeric Data Representation: The model relies on a custom text representation of numeric data, which might have limitations in capturing complex patterns present in the original numeric data.
+Training Data Source: The model is trained on MIMIC IV numeric data, and its performance might be influenced by the characteristics and biases present in that dataset.
+
+### Contact Information
+For inquiries or additional information, please contact:
+
+David Restrepo
+[email protected]
+MIT Critical Data
+
 ---
 license: mit
 ---
+