2024-02-13,14:16:04 | INFO | Running with a single process. Device cuda:0.
2024-02-13,14:16:04 | INFO | Loaded ViT-B-32 model config.
2024-02-13,14:16:07 | INFO | Loading pretrained ViT-B-32 weights (laion2b_s34b_b79k).
2024-02-13,14:16:08 | INFO | Model:
2024-02-13,14:16:08 | INFO | CLIP(
  (visual): VisionTransformer(
    (conv1): Conv2d(3, 768, kernel_size=(32, 32), stride=(32, 32), bias=False)
    (patch_dropout): Identity()
    (ln_pre): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
    (transformer): Transformer(
      (resblocks): ModuleList(
        (0-11): 12 x ResidualAttentionBlock(
          (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
          (attn): MultiheadAttention(
            (out_proj): NonDynamicallyQuantizableLinear(in_features=768, out_features=768, bias=True)
          )
          (ls_1): Identity()
          (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
          (mlp): Sequential(
            (c_fc): Linear(in_features=768, out_features=3072, bias=True)
            (gelu): GELU(approximate='none')
            (c_proj): Linear(in_features=3072, out_features=768, bias=True)
          )
          (ls_2): Identity()
        )
      )
    )
    (ln_post): LayerNorm((768,), eps=1e-05, elementwise_affine=True)
  )
  (transformer): Transformer(
    (resblocks): ModuleList(
      (0-11): 12 x ResidualAttentionBlock(
        (ln_1): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (attn): MultiheadAttention(
          (out_proj): NonDynamicallyQuantizableLinear(in_features=512, out_features=512, bias=True)
        )
        (ls_1): Identity()
        (ln_2): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
        (mlp): Sequential(
          (c_fc): Linear(in_features=512, out_features=2048, bias=True)
          (gelu): GELU(approximate='none')
          (c_proj): Linear(in_features=2048, out_features=512, bias=True)
        )
        (ls_2): Identity()
      )
    )
  )
  (token_embedding): Embedding(49408, 512)
  (ln_final): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
)
2024-02-13,14:16:08 | INFO | Params:
2024-02-13,14:16:08 | INFO |   accum_freq: 1
2024-02-13,14:16:08 | INFO |   aug_cfg: {}
2024-02-13,14:16:08 | INFO |   batch_size: 512
2024-02-13,14:16:08 | INFO |   beta1: 0.9
2024-02-13,14:16:08 | INFO |   beta2: 0.98
2024-02-13,14:16:08 | INFO |   checkpoint_path: ./logs/2024_02_13-14_16_04-model_ViT-B-32-lr_5e-05-b_512-j_8-p_amp_bf16/checkpoints
2024-02-13,14:16:08 | INFO |   coca_caption_loss_weight: 2.0
2024-02-13,14:16:08 | INFO |   coca_contrastive_loss_weight: 1.0
2024-02-13,14:16:08 | INFO |   copy_codebase: False
2024-02-13,14:16:08 | INFO |   csv_caption_key: captions
2024-02-13,14:16:08 | INFO |   csv_img_key: images
2024-02-13,14:16:08 | INFO |   csv_separator: 	
2024-02-13,14:16:08 | INFO |   dataset_resampled: False
2024-02-13,14:16:08 | INFO |   dataset_type: auto
2024-02-13,14:16:08 | INFO |   ddp_static_graph: True
2024-02-13,14:16:08 | INFO |   debug: False
2024-02-13,14:16:08 | INFO |   delete_previous_checkpoint: False
2024-02-13,14:16:08 | INFO |   device: cuda:0
2024-02-13,14:16:08 | INFO |   dist_backend: nccl
2024-02-13,14:16:08 | INFO |   dist_url: env://
2024-02-13,14:16:08 | INFO |   distill: False
2024-02-13,14:16:08 | INFO |   distill_model: None
2024-02-13,14:16:08 | INFO |   distill_pretrained: None
2024-02-13,14:16:08 | INFO |   distributed: False
2024-02-13,14:16:08 | INFO |   epochs: 15
2024-02-13,14:16:08 | INFO |   epochs_cooldown: None
2024-02-13,14:16:08 | INFO |   eps: 1e-06
2024-02-13,14:16:08 | INFO |   force_custom_text: False
2024-02-13,14:16:08 | INFO |   force_image_size: None
2024-02-13,14:16:08 | INFO |   force_patch_dropout: None
2024-02-13,14:16:08 | INFO |   force_quick_gelu: False
2024-02-13,14:16:08 | INFO |   gather_with_grad: True
2024-02-13,14:16:08 | INFO |   grad_checkpointing: False
2024-02-13,14:16:08 | INFO |   grad_clip_norm: None
2024-02-13,14:16:08 | INFO |   horovod: False
2024-02-13,14:16:08 | INFO |   image_interpolation: None
2024-02-13,14:16:08 | INFO |   image_mean: None
2024-02-13,14:16:08 | INFO |   image_resize_mode: None
2024-02-13,14:16:08 | INFO |   image_std: None
2024-02-13,14:16:08 | INFO |   imagenet_v2: None
2024-02-13,14:16:08 | INFO |   imagenet_val: None
2024-02-13,14:16:08 | INFO |   local_loss: True
2024-02-13,14:16:08 | INFO |   local_rank: 0
2024-02-13,14:16:08 | INFO |   lock_image: False
2024-02-13,14:16:08 | INFO |   lock_image_freeze_bn_stats: False
2024-02-13,14:16:08 | INFO |   lock_image_unlocked_groups: 0
2024-02-13,14:16:08 | INFO |   lock_text: False
2024-02-13,14:16:08 | INFO |   lock_text_freeze_layer_norm: False
2024-02-13,14:16:08 | INFO |   lock_text_unlocked_layers: 0
2024-02-13,14:16:08 | INFO |   log_every_n_steps: 100
2024-02-13,14:16:08 | INFO |   log_level: 20
2024-02-13,14:16:08 | INFO |   log_local: False
2024-02-13,14:16:08 | INFO |   log_path: ./logs/2024_02_13-14_16_04-model_ViT-B-32-lr_5e-05-b_512-j_8-p_amp_bf16/out.log
2024-02-13,14:16:08 | INFO |   logs: ./logs/
2024-02-13,14:16:08 | INFO |   lr: 5e-05
2024-02-13,14:16:08 | INFO |   lr_cooldown_end: 0.0
2024-02-13,14:16:08 | INFO |   lr_cooldown_power: 1.0
2024-02-13,14:16:08 | INFO |   lr_scheduler: cosine
2024-02-13,14:16:08 | INFO |   model: ViT-B-32
2024-02-13,14:16:08 | INFO |   name: 2024_02_13-14_16_04-model_ViT-B-32-lr_5e-05-b_512-j_8-p_amp_bf16
2024-02-13,14:16:08 | INFO |   no_set_device_rank: False
2024-02-13,14:16:08 | INFO |   precision: amp_bf16
2024-02-13,14:16:08 | INFO |   pretrained: laion2b_s34b_b79k
2024-02-13,14:16:08 | INFO |   pretrained_image: False
2024-02-13,14:16:08 | INFO |   rank: 0
2024-02-13,14:16:08 | INFO |   remote_sync: None
2024-02-13,14:16:08 | INFO |   remote_sync_frequency: 300
2024-02-13,14:16:08 | INFO |   remote_sync_protocol: s3
2024-02-13,14:16:08 | INFO |   report_to: 
2024-02-13,14:16:08 | INFO |   resume: None
2024-02-13,14:16:08 | INFO |   save_frequency: 5
2024-02-13,14:16:08 | INFO |   save_most_recent: False
2024-02-13,14:16:08 | INFO |   seed: 0
2024-02-13,14:16:08 | INFO |   siglip: False
2024-02-13,14:16:08 | INFO |   skip_scheduler: False
2024-02-13,14:16:08 | INFO |   tensorboard: False
2024-02-13,14:16:08 | INFO |   tensorboard_path: 
2024-02-13,14:16:08 | INFO |   torchcompile: False
2024-02-13,14:16:08 | INFO |   torchscript: False
2024-02-13,14:16:08 | INFO |   trace: False
2024-02-13,14:16:08 | INFO |   train_data: /staging/jzhang2427/flickr30k_entities/train_data/train_data_pos_neg_clip.csv
2024-02-13,14:16:08 | INFO |   train_data_upsampling_factors: None
2024-02-13,14:16:08 | INFO |   train_num_samples: None
2024-02-13,14:16:08 | INFO |   use_bn_sync: False
2024-02-13,14:16:08 | INFO |   use_bnb_linear: None
2024-02-13,14:16:08 | INFO |   val_data: None
2024-02-13,14:16:08 | INFO |   val_frequency: 5
2024-02-13,14:16:08 | INFO |   val_num_samples: None
2024-02-13,14:16:08 | INFO |   wandb: False
2024-02-13,14:16:08 | INFO |   wandb_notes: 
2024-02-13,14:16:08 | INFO |   wandb_project_name: open-clip
2024-02-13,14:16:08 | INFO |   warmup: 1024
2024-02-13,14:16:08 | INFO |   wd: 0.2
2024-02-13,14:16:08 | INFO |   workers: 8
2024-02-13,14:16:08 | INFO |   world_size: 1
2024-02-13,14:16:08 | INFO |   zeroshot_frequency: 5
2024-02-13,14:16:08 | INFO | Start epoch 0
2024-02-13,14:16:23 | INFO | Train Epoch: 0 [ 1024/45368 (1%)] Data (t): 5.578 Batch (t): 15.287, 33.4927/s, 33.4927/s/gpu LR: 0.000000 Logit Scale: 100.000 Contrastive_loss: 5.5776 (5.5776) Loss: 5.5776 (5.5776)
2024-02-13,14:17:12 | INFO | Train Epoch: 0 [90112/45368 (100%)] Data (t): 0.001 Batch (t): 0.561, 920.721/s, 920.721/s/gpu LR: 0.000004 Logit Scale: 99.995 Contrastive_loss: 3.0440 (4.3108) Loss: 3.0440 (4.3108)
2024-02-13,14:17:12 | INFO | Start epoch 1
2024-02-13,14:17:19 | INFO | Train Epoch: 1 [ 1024/45368 (1%)] Data (t): 6.202 Batch (t): 6.546, 78.2156/s, 78.2156/s/gpu LR: 0.000004 Logit Scale: 99.995 Contrastive_loss: 3.0290 (3.0290) Loss: 3.0290 (3.0290)
2024-02-13,14:18:08 | INFO | Train Epoch: 1 [90112/45368 (100%)] Data (t): 0.003 Batch (t): 0.558, 923.024/s, 923.024/s/gpu LR: 0.000009 Logit Scale: 99.994 Contrastive_loss: 2.4762 (2.7526) Loss: 2.4762 (2.7526)
2024-02-13,14:18:08 | INFO | Start epoch 2
2024-02-13,14:18:15 | INFO | Train Epoch: 2 [ 1024/45368 (1%)] Data (t): 5.969 Batch (t): 6.301, 81.2608/s, 81.2608/s/gpu LR: 0.000009 Logit Scale: 99.994 Contrastive_loss: 2.1421 (2.1421) Loss: 2.1421 (2.1421)
2024-02-13,14:19:03 | INFO | Train Epoch: 2 [90112/45368 (100%)] Data (t): 0.003 Batch (t): 0.559, 866.204/s, 866.204/s/gpu LR: 0.000013 Logit Scale: 99.992 Contrastive_loss: 1.9024 (2.0222) Loss: 1.9024 (2.0222)
2024-02-13,14:19:04 | INFO | Start epoch 3
2024-02-13,14:19:12 | INFO | Train Epoch: 3 [ 1024/45368 (1%)] Data (t): 7.652 Batch (t): 7.986, 64.1139/s, 64.1139/s/gpu LR: 0.000013 Logit Scale: 99.992 Contrastive_loss: 1.5644 (1.5644) Loss: 1.5644 (1.5644)
2024-02-13,14:20:03 | INFO | Train Epoch: 3 [90112/45368 (100%)] Data (t): 0.052 Batch (t): 0.583, 922.873/s, 922.873/s/gpu LR: 0.000017 Logit Scale: 99.990 Contrastive_loss: 1.5900 (1.5772) Loss: 1.5900 (1.5772)
2024-02-13,14:20:03 | INFO | Start epoch 4
2024-02-13,14:20:10 | INFO | Train Epoch: 4 [ 1024/45368 (1%)] Data (t): 6.206 Batch (t): 6.543, 78.2560/s, 78.2560/s/gpu LR: 0.000017 Logit Scale: 99.990 Contrastive_loss: 1.1582 (1.1582) Loss: 1.1582 (1.1582)
2024-02-13,14:21:00 | INFO | Train Epoch: 4 [90112/45368 (100%)] Data (t): 0.026 Batch (t): 0.573, 865.168/s, 865.168/s/gpu LR: 0.000021 Logit Scale: 99.986 Contrastive_loss: 1.1860 (1.1721) Loss: 1.1860 (1.1721)
2024-02-13,14:21:02 | INFO | Start epoch 5
2024-02-13,14:21:08 | INFO | Train Epoch: 5 [ 1024/45368 (1%)] Data (t): 6.005 Batch (t): 6.350, 80.6259/s, 80.6259/s/gpu LR: 0.000022 Logit Scale: 99.985 Contrastive_loss: 0.90302 (0.90302) Loss: 0.90302 (0.90302)
2024-02-13,14:21:57 | INFO | Train Epoch: 5 [90112/45368 (100%)] Data (t): 0.003 Batch (t): 0.558, 923.267/s, 923.267/s/gpu LR: 0.000026 Logit Scale: 99.970 Contrastive_loss: 1.0292 (0.96613) Loss: 1.0292 (0.96613)
2024-02-13,14:21:57 | INFO | Start epoch 6
2024-02-13,14:22:04 | INFO | Train Epoch: 6 [ 1024/45368 (1%)] Data (t): 6.185 Batch (t): 6.521, 78.5109/s, 78.5109/s/gpu LR: 0.000026 Logit Scale: 99.969 Contrastive_loss: 0.77590 (0.77590) Loss: 0.77590 (0.77590)
2024-02-13,14:22:53 | INFO | Train Epoch: 6 [90112/45368 (100%)] Data (t): 0.007 Batch (t): 0.560, 921.958/s, 921.958/s/gpu LR: 0.000030 Logit Scale: 99.916 Contrastive_loss: 0.92965 (0.85277) Loss: 0.92965 (0.85277)
2024-02-13,14:22:53 | INFO | Start epoch 7
2024-02-13,14:22:59 | INFO | Train Epoch: 7 [ 1024/45368 (1%)] Data (t): 5.502 Batch (t): 5.834, 87.7655/s, 87.7655/s/gpu LR: 0.000030 Logit Scale: 99.915 Contrastive_loss: 0.72926 (0.72926) Loss: 0.72926 (0.72926)
2024-02-13,14:23:49 | INFO | Train Epoch: 7 [90112/45368 (100%)] Data (t): 0.026 Batch (t): 0.574, 911.691/s, 911.691/s/gpu LR: 0.000034 Logit Scale: 99.846 Contrastive_loss: 0.86592 (0.79759) Loss: 0.86592 (0.79759)
2024-02-13,14:23:49 | INFO | Start epoch 8
2024-02-13,14:23:56 | INFO | Train Epoch: 8 [ 1024/45368 (1%)] Data (t): 6.269 Batch (t): 6.609, 77.4675/s, 77.4675/s/gpu LR: 0.000034 Logit Scale: 99.846 Contrastive_loss: 0.57734 (0.57734) Loss: 0.57734 (0.57734)
2024-02-13,14:24:45 | INFO | Train Epoch: 8 [90112/45368 (100%)] Data (t): 0.004 Batch (t): 0.559, 890.899/s, 890.899/s/gpu LR: 0.000039 Logit Scale: 99.772 Contrastive_loss: 0.62928 (0.60331) Loss: 0.62928 (0.60331)
2024-02-13,14:24:45 | INFO | Start epoch 9
2024-02-13,14:24:51 | INFO | Train Epoch: 9 [ 1024/45368 (1%)] Data (t): 5.301 Batch (t): 5.647, 90.6720/s, 90.6720/s/gpu LR: 0.000039 Logit Scale: 99.772 Contrastive_loss: 0.53649 (0.53649) Loss: 0.53649 (0.53649)
2024-02-13,14:25:41 | INFO | Train Epoch: 9 [90112/45368 (100%)] Data (t): 0.026 Batch (t): 0.571, 864.281/s, 864.281/s/gpu LR: 0.000043 Logit Scale: 99.671 Contrastive_loss: 0.61911 (0.57780) Loss: 0.61911 (0.57780)
2024-02-13,14:25:43 | INFO | Start epoch 10
2024-02-13,14:25:50 | INFO | Train Epoch: 10 [ 1024/45368 (1%)] Data (t): 6.521 Batch (t): 6.853, 74.7106/s, 74.7106/s/gpu LR: 0.000043 Logit Scale: 99.669 Contrastive_loss: 0.48743 (0.48743) Loss: 0.48743 (0.48743)
2024-02-13,14:26:39 | INFO | Train Epoch: 10 [90112/45368 (100%)] Data (t): 0.015 Batch (t): 0.566, 922.642/s, 922.642/s/gpu LR: 0.000047 Logit Scale: 99.527 Contrastive_loss: 0.58032 (0.53388) Loss: 0.58032 (0.53388)
2024-02-13,14:26:40 | INFO | Start epoch 11
2024-02-13,14:26:47 | INFO | Train Epoch: 11 [ 1024/45368 (1%)] Data (t): 7.228 Batch (t): 7.567, 67.6600/s, 67.6600/s/gpu LR: 0.000047 Logit Scale: 99.525 Contrastive_loss: 0.38881 (0.38881) Loss: 0.38881 (0.38881)
2024-02-13,14:27:37 | INFO | Train Epoch: 11 [90112/45368 (100%)] Data (t): 0.025 Batch (t): 0.568, 915.143/s, 915.143/s/gpu LR: 0.000049 Logit Scale: 99.360 Contrastive_loss: 0.47798 (0.43340) Loss: 0.47798 (0.43340)
2024-02-13,14:27:37 | INFO | Start epoch 12
2024-02-13,14:27:44 | INFO | Train Epoch: 12 [ 1024/45368 (1%)] Data (t): 6.610 Batch (t): 6.942, 73.7587/s, 73.7587/s/gpu LR: 0.000049 Logit Scale: 99.357 Contrastive_loss: 0.34789 (0.34789) Loss: 0.34789 (0.34789)
2024-02-13,14:28:33 | INFO | Train Epoch: 12 [90112/45368 (100%)] Data (t): 0.009 Batch (t): 0.563, 923.298/s, 923.298/s/gpu LR: 0.000033 Logit Scale: 99.254 Contrastive_loss: 0.32083 (0.33436) Loss: 0.32083 (0.33436)
2024-02-13,14:28:33 | INFO | Start epoch 13
2024-02-13,14:28:39 | INFO | Train Epoch: 13 [ 1024/45368 (1%)] Data (t): 5.156 Batch (t): 5.490, 93.2544/s, 93.2544/s/gpu LR: 0.000032 Logit Scale: 99.253 Contrastive_loss: 0.23794 (0.23794) Loss: 0.23794 (0.23794)
2024-02-13,14:29:29 | INFO | Train Epoch: 13 [90112/45368 (100%)] Data (t): 0.020 Batch (t): 0.574, 922.599/s, 922.599/s/gpu LR: 0.000010 Logit Scale: 99.241 Contrastive_loss: 0.13853 (0.18823) Loss: 0.13853 (0.18823)
2024-02-13,14:29:30 | INFO | Start epoch 14
2024-02-13,14:29:36 | INFO | Train Epoch: 14 [ 1024/45368 (1%)] Data (t): 5.908 Batch (t): 6.247, 81.9657/s, 81.9657/s/gpu LR: 0.000010 Logit Scale: 99.241 Contrastive_loss: 0.16239 (0.16239) Loss: 0.16239 (0.16239)
2024-02-13,14:30:25 | INFO | Train Epoch: 14 [90112/45368 (100%)] Data (t): 0.004 Batch (t): 0.561, 888.267/s, 888.267/s/gpu LR: 0.000000 Logit Scale: 99.249 Contrastive_loss: 0.11300 (0.13770) Loss: 0.11300 (0.13770)