{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "afb0bfd9-f789-4c69-ab8a-286b34e5b412",
   "metadata": {},
   "source": [
    "# 2.2 从头训练dna gpt2大模型"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12b19a52-64cb-4ccd-83f6-1fb37422c68d",
   "metadata": {},
   "source": [
    "### GPT-2 模型简介\n",
    "\n",
    "GPT-2（Generative Pre-trained Transformer 2）是由 OpenAI 开发的一种基于 Transformer 架构的大型语言模型。它是一种生成式模型，意味着它可以生成连贯且上下文相关的文本。GPT-2 通过大量的无监督预训练数据学习了语言的统计规律，从而能够在多种自然语言处理任务中表现出色，如文本生成、问答、翻译等。\n",
    "\n",
    "### 参数规模\n",
    "\n",
    "GPT-2 提供了多个不同大小的版本，每个版本都有不同的参数量和计算资源需求。以下是主要版本及其参数规模：\n",
    "\n",
    "1. **GPT-2 Small**：\n",
    "   - 参数量：1.54 亿个参数\n",
    "   - 层数：12 层\n",
    "   - 隐藏层维度：768\n",
    "   - 注意力头数：12\n",
    "\n",
    "2. **GPT-2 Medium**：\n",
    "   - 参数量：3.45 亿个参数\n",
    "   - 层数：24 层\n",
    "   - 隐藏层维度：1024\n",
    "   - 注意力头数：16\n",
    "\n",
    "3. **GPT-2 Large**：\n",
    "   - 参数量：7.74 亿个参数\n",
    "   - 层数：36 层\n",
    "   - 隐藏层维度：1280\n",
    "   - 注意力头数：20\n",
    "\n",
    "4. **GPT-2 XL**：\n",
    "   - 参数量：15.58 亿个参数\n",
    "   - 层数：48 层\n",
    "   - 隐藏层维度：1600\n",
    "   - 注意力头数：25\n",
    "\n",
    "这些不同规模的模型可以根据具体的硬件资源和应用场景选择使用。较小的模型适合资源有限的情况，而较大的模型则能提供更强大的生成能力和更好的性能。\n",
    "\n",
    "### 历史背景\n",
    "\n",
    "- **发布日期**：GPT-2 最初于 2019 年 2 月发布。OpenAI 在最初并没有一次性公开所有版本，而是逐步发布了不同规模的模型。\n",
    "  \n",
    "- **开发动机**：GPT-2 是在 GPT-1 的基础上进行的重大改进。它引入了更多的参数和更大的训练数据集，显著提升了模型的能力。此外，GPT-2 还展示了强大的文本生成能力，甚至能够生成逼真的文章段落，这引发了关于 AI 伦理和社会影响的广泛讨论。\n",
    "\n",
    "- **伦理考虑**：由于 GPT-2 的强大生成能力，OpenAI 初始时对模型的发布采取了谨慎的态度，担心其可能被滥用（例如用于生成假新闻或恶意内容）。因此，他们选择了分阶段发布，并进行了广泛的伦理讨论和研究。\n",
    "\n",
    "- **后续发展**：随着技术的进步和社区的反馈，OpenAI 最终发布了完整的 GPT-2 模型，并开放了 API 接口。此后，许多研究人员和开发者基于 GPT-2 进行了各种创新应用，包括但不限于对话系统、自动摘要、机器翻译等。\n",
    "\n",
    "- **后续版本**：GPT-2 的成功为后来的 GPT-3 和其他大模型的发展奠定了基础。GPT-3 进一步增加了参数量，达到了 1750 亿个参数，极大地提高了模型的表现力和多功能性。\n",
    "\n",
    "### DNA GPT2\n",
    "\n",
    "GPT-2 是一个重要的里程碑，在自然语言处理领域展现了强大的文本生成和理解能力。它的发布不仅推动了深度学习技术的发展,通过不断迭代和改进，GPT 系列模型已经成为现代 NLP 应用的核心工具之一。这里我们就使用GPT2网络，训练一个DNA的GPT2模型。\n",
    "\n",
    "### 网络结构\n",
    "GPT2 small的基本网络结构如下所示：\n",
    "\n",
    "\n",
    "<img src=\"img/gpt2-stru.png\" width=\"600px\" />"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "70581590-096f-45f8-b13b-b84e88615849",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import AutoTokenizer, GPT2LMHeadModel, AutoConfig,GPT2Tokenizer\n",
    "from transformers import GPT2Tokenizer,GPT2Model,AutoModel\n",
    "from transformers import DataCollatorForLanguageModeling\n",
    "from transformers import Trainer, TrainingArguments\n",
    "from transformers import LineByLineTextDataset\n",
    "from tokenizers import Tokenizer\n",
    "from datasets import load_dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3b42fc2-8b6d-40c0-850e-a72271f62d32",
   "metadata": {},
   "source": [
    "首先是分词器，我们使用的是我们上一节训练的dna_bpe_dict分词器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "34bd2710-fc99-4eda-9364-343f62f56b1e",
   "metadata": {},
   "outputs": [],
   "source": [
    "#tokenizer = GPT2Tokenizer.from_pretrained(\"dnagpt/dna_bpe_dict_1g\")\n",
    "tokenizer = GPT2Tokenizer.from_pretrained(\"dna_bpe_dict\")\n",
    "tokenizer.pad_token = tokenizer.eos_token"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c59af3e-889e-43f5-962a-d12c5ecee347",
   "metadata": {},
   "source": [
    "然后是模型的配置，我们采用的是从头训练的方法\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "87435829-f522-4820-a51d-11fa4afee6d7",
   "metadata": {},
   "outputs": [],
   "source": [
    "max_length = 256 #最大输入长度\n",
    "\n",
    "config = AutoConfig.from_pretrained(\n",
    "    \"gpt2\",\n",
    "    vocab_size=len(tokenizer),\n",
    "    n_ctx=max_length, #最大长度\n",
    "    bos_token_id=tokenizer.bos_token_id,\n",
    "    eos_token_id=tokenizer.eos_token_id,\n",
    ")\n",
    "\n",
    "model = GPT2LMHeadModel(config) #for pretrain,从头预训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a0adfdd-4be9-4027-a12d-3bf848be3012",
   "metadata": {},
   "outputs": [],
   "source": [
    "接着是训练数据集，"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6a6bbb6d-8f7d-4652-8961-bffc0466beaf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "4e8e73b13d42451bb7214b50bf1d8d47",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Generating train split: 0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6a3e649a8ca14fc8adfc361c2f6eeb7e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map (num_proc=15):   0%|          | 0/971635 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b0fb7862bca842518fa4e96901b93be4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map (num_proc=15):   0%|          | 0/107960 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 1. load dna dataset\n",
    "raw_dataset = load_dataset('text',  data_files=\"../01-data_env/data/dna_1g.txt\")\n",
    "dataset = raw_dataset[\"train\"].train_test_split(test_size=0.1, shuffle=True)\n",
    "\n",
    "# 2. tokenize\n",
    "def tokenize_function(examples):\n",
    "    return tokenizer(examples['text'], truncation=True, padding='max_length', max_length=max_length)\n",
    "\n",
    "# 3. 对数据集应用分词函数\n",
    "tokenized_datasets = dataset.map(tokenize_function, batched=True, remove_columns=['text'], num_proc=15)  # 设置为你的 CPU 核心数或根据需要调整\n",
    "\n",
    "# 4. 创建一个数据收集器，用于动态填充和遮蔽\n",
    "data_collator = DataCollatorForLanguageModeling(\n",
    "    tokenizer=tokenizer, mlm=False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "9c69b1e1-bcc1-4dea-8db4-6df52f96d9de",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2024-12-25 01:00:37,579] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/miniconda3/compiler_compat/ld: cannot find -laio: No such file or directory\n",
      "collect2: error: ld returned 1 exit status\n",
      "/root/miniconda3/compiler_compat/ld: warning: libpthread.so.0, needed by /usr/local/cuda/lib64/libcufile.so, not found (try using -rpath or -rpath-link)\n",
      "/root/miniconda3/compiler_compat/ld: warning: libstdc++.so.6, needed by /usr/local/cuda/lib64/libcufile.so, not found (try using -rpath or -rpath-link)\n",
      "/root/miniconda3/compiler_compat/ld: warning: libm.so.6, needed by /usr/local/cuda/lib64/libcufile.so, not found (try using -rpath or -rpath-link)\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::runtime_error::~runtime_error()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__gxx_personality_v0@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::tellp()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::chrono::_V2::steady_clock::now()@GLIBCXX_3.4.19'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_M_replace_aux(unsigned long, unsigned long, unsigned long, char)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for bool@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_logic_error(char const*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::logic_error@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::~locale()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&, unsigned long, unsigned long)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_end_catch@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_ofstream<char, std::char_traits<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::logic_error::~logic_error()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for __cxxabiv1::__si_class_type_info@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios<char, std::char_traits<char> >::_M_cache_locale(std::locale const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator new[](unsigned long)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_M_leak_hard()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ifstream<char, std::char_traits<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf<wchar_t, std::char_traits<wchar_t> >::basic_streambuf(std::basic_streambuf<wchar_t, std::char_traits<wchar_t> > const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::append(char const*, unsigned long)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned short@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::resize(unsigned long, char)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for char const*@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ctype<char>::_M_widen_init() const@GLIBCXX_3.4.11'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_invalid_argument(char const*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::operator=(std::locale const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios<wchar_t, std::char_traits<wchar_t> >::_M_cache_locale(std::locale const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_decrement(std::_Rb_tree_node_base const*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_free_exception@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::condition_variable::notify_one()@GLIBCXX_3.4.11'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::Init::~Init()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_pure_virtual@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::flush()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for __cxxabiv1::__class_type_info@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_rethrow@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_fstream<char, std::char_traits<char> >::~basic_fstream()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::compare(char const*) const@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_ostringstream<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::locale()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::chrono::_V2::system_clock::now()@GLIBCXX_3.4.19'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_ifstream<char, std::char_traits<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Hash_bytes(void const*, unsigned long, unsigned long)@CXXABI_1.3.5'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert<long long>(long long)@GLIBCXX_3.4.9'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for char*@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_Prime_rehash_policy::_M_need_rehash(unsigned long, unsigned long, unsigned long) const@GLIBCXX_3.4.18'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::out_of_range@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert<unsigned long>(unsigned long)@GLIBCXX_3.4.9'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_increment(std::_Rb_tree_node_base const*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::~ios_base()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::range_error::~range_error()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__basic_file<char>::~__basic_file()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_guard_acquire@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert<bool>(bool)@GLIBCXX_3.4.9'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::overflow_error@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_fstream<char, std::char_traits<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::range_error@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ios<char, std::char_traits<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_filebuf<char, std::char_traits<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator delete[](void*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(unsigned long, char, std::allocator<char> const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_List_node_base::_M_transfer(std::__detail::_List_node_base*, std::__detail::_List_node_base*)@GLIBCXX_3.4.15'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::replace(unsigned long, unsigned long, char const*, unsigned long)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for std::exception@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::_Rep::_M_destroy(std::allocator<wchar_t> const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::istream& std::istream::_M_extract<double>(double&)@GLIBCXX_3.4.9'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf<char, std::char_traits<char> >::close()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_fstream<char, std::char_traits<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ifstream<char, std::char_traits<char> >::basic_ifstream(char const*, std::_Ios_Openmode)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::append(std::string const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator new(unsigned long)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_istringstream<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned int@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::append(char const*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::domain_error@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::find(char, unsigned long) const@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::put(char)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for int@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_bad_alloc()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_thread_atexit@CXXABI_1.3.7'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_increment(std::_Rb_tree_node_base*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ifstream<char, std::char_traits<char> >::~basic_ifstream()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::Init::Init()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::condition_variable::condition_variable()@GLIBCXX_3.4.11'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf<char, std::char_traits<char> >::basic_filebuf()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_istringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::domain_error::~domain_error()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::cerr@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::find(char const*, unsigned long, unsigned long) const@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_istringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::allocator<char> const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::str() const@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::invalid_argument@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for void*@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::assign(std::string const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >::~basic_ostringstream()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned long@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*)@GLIBCXX_3.4.15'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__detail::_List_node_base::_M_unhook()@GLIBCXX_3.4.15'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ostringstream<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::_M_sync(char*, unsigned long, unsigned long)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_iostream<char, std::char_traits<char> >::~basic_iostream()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::locale::locale(std::locale const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_istringstream<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `log2f@GLIBC_2.2.5'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::operator<<(std::basic_streambuf<char, std::char_traits<char> >*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_streambuf<wchar_t, std::char_traits<wchar_t> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::exception::~exception()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__basic_file<char>::is_open() const@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_istringstream<char, std::char_traits<char>, std::allocator<char> >::~basic_istringstream()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::swap(std::string&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned long*@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf<char, std::char_traits<char> >::basic_streambuf(std::basic_streambuf<char, std::char_traits<char> > const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios<char, std::char_traits<char> >::init(std::basic_streambuf<char, std::char_traits<char> >*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_bad_cast()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios<char, std::char_traits<char> >::clear(std::_Ios_Iostate)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf<wchar_t, std::char_traits<wchar_t> >::operator=(std::basic_streambuf<wchar_t, std::char_traits<wchar_t> > const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `operator delete(void*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream::operator<<(int)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_Rep::_S_empty_rep_storage@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_Rep::_M_destroy(std::allocator<char> const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_iostream<wchar_t, std::char_traits<wchar_t> >::~basic_iostream()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::runtime_error@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ofstream<char, std::char_traits<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >::~basic_stringstream()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `VTT for std::basic_stringstream<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert<long>(long)@GLIBCXX_3.4.9'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::istream::get()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned long long@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::out_of_range::~out_of_range()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::length_error::~length_error()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long)@GLIBCXX_3.4.9'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::invalid_argument::~invalid_argument()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::swap(std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::cout@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert<unsigned long long>(unsigned long long)@GLIBCXX_3.4.9'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for int*@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert<void const*>(void const*)@GLIBCXX_3.4.9'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::underflow_error@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_streambuf<char, std::char_traits<char> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for std::out_of_range@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_allocate_exception@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_ios<wchar_t, std::char_traits<wchar_t> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for void const*@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ios<wchar_t, std::char_traits<wchar_t> >::init(std::basic_streambuf<wchar_t, std::char_traits<wchar_t> >*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::reserve(unsigned long)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_begin_catch@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for long@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::_Rep::_S_empty_rep_storage@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::_M_leak()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_stringbuf<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::_M_sync(wchar_t*, unsigned long, unsigned long)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::istream::getline(char*, long, char)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_istream<char, std::char_traits<char> >& std::getline<char, std::char_traits<char>, std::allocator<char> >(std::basic_istream<char, std::char_traits<char> >&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, char)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringstream<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::condition_variable::~condition_variable()@GLIBCXX_3.4.11'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::basic_stringbuf<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::insert(unsigned long, char const*, unsigned long)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::string::assign(char const*, unsigned long)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for unsigned char@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ios_base::ios_base()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_out_of_range(char const*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::overflow_error::~overflow_error()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_length_error(char const*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::__throw_system_error(int)@GLIBCXX_3.4.11'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ofstream<char, std::char_traits<char> >::close()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::ostream& std::ostream::_M_insert<double>(double)@GLIBCXX_3.4.9'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_streambuf<char, std::char_traits<char> >::operator=(std::basic_streambuf<char, std::char_traits<char> > const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `typeinfo for long long@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_ifstream<char, std::char_traits<char> >::close()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_guard_release@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `__cxa_throw@CXXABI_1.3'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::underflow_error::~underflow_error()@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::_Rb_tree_decrement(std::_Rb_tree_node_base*)@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `vtable for std::length_error@GLIBCXX_3.4'\n",
      "/root/miniconda3/compiler_compat/ld: /usr/local/cuda/lib64/libcufile.so: undefined reference to `std::basic_filebuf<char, std::char_traits<char> >::~basic_filebuf()@GLIBCXX_3.4'\n",
      "collect2: error: ld returned 1 exit status\n"
     ]
    }
   ],
   "source": [
    "run_path = \"gpt2_run\"\n",
    "train_epoches = 5\n",
    "batch_size = 10\n",
    "\n",
    "\n",
    "training_args = TrainingArguments(\n",
    "        output_dir=run_path,\n",
    "        overwrite_output_dir=True,\n",
    "        num_train_epochs=train_epoches,\n",
    "        per_device_train_batch_size=batch_size,\n",
    "        save_steps=2000,\n",
    "        save_total_limit=2,\n",
    "        prediction_loss_only=True,\n",
    "        fp16=True, #v100没法用\n",
    "    )\n",
    "\n",
    "\n",
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=training_args,\n",
    "    train_dataset=tokenized_datasets[\"train\"],\n",
    "    eval_dataset=tokenized_datasets[\"test\"],\n",
    "    data_collator=data_collator,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8ac66e61-4a1c-4fc2-8cde-9a022d08b990",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='485820' max='485820' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [485820/485820 10:36:19, Epoch 5/5]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>500</td>\n",
       "      <td>8.815900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1000</td>\n",
       "      <td>8.676700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1500</td>\n",
       "      <td>8.665800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2000</td>\n",
       "      <td>8.648500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2500</td>\n",
       "      <td>8.620900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3000</td>\n",
       "      <td>8.595700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3500</td>\n",
       "      <td>8.565000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4000</td>\n",
       "      <td>8.536300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4500</td>\n",
       "      <td>8.515300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5000</td>\n",
       "      <td>8.499900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5500</td>\n",
       "      <td>8.481800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6000</td>\n",
       "      <td>8.470000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6500</td>\n",
       "      <td>8.460300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7000</td>\n",
       "      <td>8.444200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7500</td>\n",
       "      <td>8.438200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8000</td>\n",
       "      <td>8.425100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8500</td>\n",
       "      <td>8.420500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9000</td>\n",
       "      <td>8.414300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9500</td>\n",
       "      <td>8.398100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10000</td>\n",
       "      <td>8.389400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>10500</td>\n",
       "      <td>8.373400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11000</td>\n",
       "      <td>8.371700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11500</td>\n",
       "      <td>8.362100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>12000</td>\n",
       "      <td>8.351400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>12500</td>\n",
       "      <td>8.346400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13000</td>\n",
       "      <td>8.337300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>13500</td>\n",
       "      <td>8.333800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>14000</td>\n",
       "      <td>8.329100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>14500</td>\n",
       "      <td>8.321000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15000</td>\n",
       "      <td>8.312400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>15500</td>\n",
       "      <td>8.301200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16000</td>\n",
       "      <td>8.297400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>16500</td>\n",
       "      <td>8.297100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>17000</td>\n",
       "      <td>8.287500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>17500</td>\n",
       "      <td>8.281200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18000</td>\n",
       "      <td>8.272700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>18500</td>\n",
       "      <td>8.272900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>19000</td>\n",
       "      <td>8.268500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>19500</td>\n",
       "      <td>8.254900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20000</td>\n",
       "      <td>8.254200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>20500</td>\n",
       "      <td>8.237700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>21000</td>\n",
       "      <td>8.236800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>21500</td>\n",
       "      <td>8.231200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>22000</td>\n",
       "      <td>8.223700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>22500</td>\n",
       "      <td>8.213700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>23000</td>\n",
       "      <td>8.207700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>23500</td>\n",
       "      <td>8.203600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24000</td>\n",
       "      <td>8.199700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>24500</td>\n",
       "      <td>8.193900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25000</td>\n",
       "      <td>8.191600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25500</td>\n",
       "      <td>8.177000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26000</td>\n",
       "      <td>8.181600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>26500</td>\n",
       "      <td>8.161000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>27000</td>\n",
       "      <td>8.160200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>27500</td>\n",
       "      <td>8.145700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>28000</td>\n",
       "      <td>8.154500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>28500</td>\n",
       "      <td>8.152100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>29000</td>\n",
       "      <td>8.130000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>29500</td>\n",
       "      <td>8.139600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>30000</td>\n",
       "      <td>8.126700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>30500</td>\n",
       "      <td>8.123800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>31000</td>\n",
       "      <td>8.116000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>31500</td>\n",
       "      <td>8.112200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>32000</td>\n",
       "      <td>8.113700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>32500</td>\n",
       "      <td>8.104200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33000</td>\n",
       "      <td>8.115900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33500</td>\n",
       "      <td>8.086900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>34000</td>\n",
       "      <td>8.089300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>34500</td>\n",
       "      <td>8.091100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>35000</td>\n",
       "      <td>8.094500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>35500</td>\n",
       "      <td>8.084900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>36000</td>\n",
       "      <td>8.071100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>36500</td>\n",
       "      <td>8.075700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>37000</td>\n",
       "      <td>8.068200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>37500</td>\n",
       "      <td>8.060400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38000</td>\n",
       "      <td>8.057600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>38500</td>\n",
       "      <td>8.056500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39000</td>\n",
       "      <td>8.062000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>39500</td>\n",
       "      <td>8.038800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40000</td>\n",
       "      <td>8.047200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>40500</td>\n",
       "      <td>8.041100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41000</td>\n",
       "      <td>8.034100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>41500</td>\n",
       "      <td>8.040300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>42000</td>\n",
       "      <td>8.038200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>42500</td>\n",
       "      <td>8.015500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>43000</td>\n",
       "      <td>8.018900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>43500</td>\n",
       "      <td>8.009800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>44000</td>\n",
       "      <td>8.020700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>44500</td>\n",
       "      <td>8.008000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>45000</td>\n",
       "      <td>8.007000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>45500</td>\n",
       "      <td>7.995600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>46000</td>\n",
       "      <td>8.010800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>46500</td>\n",
       "      <td>7.999200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>47000</td>\n",
       "      <td>7.981500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>47500</td>\n",
       "      <td>7.973400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>48000</td>\n",
       "      <td>7.983600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>48500</td>\n",
       "      <td>7.972400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>49000</td>\n",
       "      <td>7.972100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>49500</td>\n",
       "      <td>7.953700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50000</td>\n",
       "      <td>7.952200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50500</td>\n",
       "      <td>7.969300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>51000</td>\n",
       "      <td>7.963000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>51500</td>\n",
       "      <td>7.956300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>52000</td>\n",
       "      <td>7.949500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>52500</td>\n",
       "      <td>7.942300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>53000</td>\n",
       "      <td>7.955100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>53500</td>\n",
       "      <td>7.939600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>54000</td>\n",
       "      <td>7.930700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>54500</td>\n",
       "      <td>7.938300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>55000</td>\n",
       "      <td>7.928400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>55500</td>\n",
       "      <td>7.944200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>56000</td>\n",
       "      <td>7.935500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>56500</td>\n",
       "      <td>7.925200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>57000</td>\n",
       "      <td>7.939800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>57500</td>\n",
       "      <td>7.939600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>58000</td>\n",
       "      <td>7.940500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>58500</td>\n",
       "      <td>7.929200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>59000</td>\n",
       "      <td>7.908200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>59500</td>\n",
       "      <td>7.929500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>60000</td>\n",
       "      <td>7.900000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>60500</td>\n",
       "      <td>7.915500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>61000</td>\n",
       "      <td>7.898200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>61500</td>\n",
       "      <td>7.890100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>62000</td>\n",
       "      <td>7.919500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>62500</td>\n",
       "      <td>7.904000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>63000</td>\n",
       "      <td>7.894300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>63500</td>\n",
       "      <td>7.887500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>64000</td>\n",
       "      <td>7.864500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>64500</td>\n",
       "      <td>7.862500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>65000</td>\n",
       "      <td>7.875500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>65500</td>\n",
       "      <td>7.876800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>66000</td>\n",
       "      <td>7.875900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>66500</td>\n",
       "      <td>7.871200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>67000</td>\n",
       "      <td>7.866700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>67500</td>\n",
       "      <td>7.876500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>68000</td>\n",
       "      <td>7.867000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>68500</td>\n",
       "      <td>7.894700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>69000</td>\n",
       "      <td>7.865000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>69500</td>\n",
       "      <td>7.864100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>70000</td>\n",
       "      <td>7.851500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>70500</td>\n",
       "      <td>7.865400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>71000</td>\n",
       "      <td>7.865400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>71500</td>\n",
       "      <td>7.853900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>72000</td>\n",
       "      <td>7.851400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>72500</td>\n",
       "      <td>7.841300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>73000</td>\n",
       "      <td>7.855100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>73500</td>\n",
       "      <td>7.834700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>74000</td>\n",
       "      <td>7.839000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>74500</td>\n",
       "      <td>7.829300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>75000</td>\n",
       "      <td>7.827500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>75500</td>\n",
       "      <td>7.839800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>76000</td>\n",
       "      <td>7.827700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>76500</td>\n",
       "      <td>7.830800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>77000</td>\n",
       "      <td>7.847000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>77500</td>\n",
       "      <td>7.802900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>78000</td>\n",
       "      <td>7.821600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>78500</td>\n",
       "      <td>7.803700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>79000</td>\n",
       "      <td>7.826500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>79500</td>\n",
       "      <td>7.802300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>80000</td>\n",
       "      <td>7.819800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>80500</td>\n",
       "      <td>7.802800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>81000</td>\n",
       "      <td>7.823000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>81500</td>\n",
       "      <td>7.813800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>82000</td>\n",
       "      <td>7.806000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>82500</td>\n",
       "      <td>7.807700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>83000</td>\n",
       "      <td>7.805100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>83500</td>\n",
       "      <td>7.797100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>84000</td>\n",
       "      <td>7.791600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>84500</td>\n",
       "      <td>7.785800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>85000</td>\n",
       "      <td>7.788200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>85500</td>\n",
       "      <td>7.802900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>86000</td>\n",
       "      <td>7.794000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>86500</td>\n",
       "      <td>7.780700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>87000</td>\n",
       "      <td>7.795700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>87500</td>\n",
       "      <td>7.794100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>88000</td>\n",
       "      <td>7.778400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>88500</td>\n",
       "      <td>7.785000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>89000</td>\n",
       "      <td>7.792800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>89500</td>\n",
       "      <td>7.783100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>90000</td>\n",
       "      <td>7.750600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>90500</td>\n",
       "      <td>7.772600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>91000</td>\n",
       "      <td>7.786200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>91500</td>\n",
       "      <td>7.781700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>92000</td>\n",
       "      <td>7.761300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>92500</td>\n",
       "      <td>7.755300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>93000</td>\n",
       "      <td>7.757600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>93500</td>\n",
       "      <td>7.767900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>94000</td>\n",
       "      <td>7.765800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>94500</td>\n",
       "      <td>7.767900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>95000</td>\n",
       "      <td>7.753900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>95500</td>\n",
       "      <td>7.777700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>96000</td>\n",
       "      <td>7.757300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>96500</td>\n",
       "      <td>7.745300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>97000</td>\n",
       "      <td>7.749500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>97500</td>\n",
       "      <td>7.712700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>98000</td>\n",
       "      <td>7.725600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>98500</td>\n",
       "      <td>7.738500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>99000</td>\n",
       "      <td>7.720500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>99500</td>\n",
       "      <td>7.752200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>100000</td>\n",
       "      <td>7.719000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>100500</td>\n",
       "      <td>7.748300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>101000</td>\n",
       "      <td>7.741400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>101500</td>\n",
       "      <td>7.745800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>102000</td>\n",
       "      <td>7.727900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>102500</td>\n",
       "      <td>7.726500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>103000</td>\n",
       "      <td>7.717800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>103500</td>\n",
       "      <td>7.718600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>104000</td>\n",
       "      <td>7.737900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>104500</td>\n",
       "      <td>7.726600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>105000</td>\n",
       "      <td>7.711000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>105500</td>\n",
       "      <td>7.698200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>106000</td>\n",
       "      <td>7.740300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>106500</td>\n",
       "      <td>7.722700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>107000</td>\n",
       "      <td>7.707300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>107500</td>\n",
       "      <td>7.713700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>108000</td>\n",
       "      <td>7.720900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>108500</td>\n",
       "      <td>7.713000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>109000</td>\n",
       "      <td>7.702100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>109500</td>\n",
       "      <td>7.710200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>110000</td>\n",
       "      <td>7.714700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>110500</td>\n",
       "      <td>7.710000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>111000</td>\n",
       "      <td>7.704200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>111500</td>\n",
       "      <td>7.675000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>112000</td>\n",
       "      <td>7.708400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>112500</td>\n",
       "      <td>7.693200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>113000</td>\n",
       "      <td>7.690100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>113500</td>\n",
       "      <td>7.689600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>114000</td>\n",
       "      <td>7.708600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>114500</td>\n",
       "      <td>7.685800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>115000</td>\n",
       "      <td>7.681600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>115500</td>\n",
       "      <td>7.669200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>116000</td>\n",
       "      <td>7.708900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>116500</td>\n",
       "      <td>7.685400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>117000</td>\n",
       "      <td>7.675700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>117500</td>\n",
       "      <td>7.663500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>118000</td>\n",
       "      <td>7.694700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>118500</td>\n",
       "      <td>7.696000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>119000</td>\n",
       "      <td>7.687400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>119500</td>\n",
       "      <td>7.679600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>120000</td>\n",
       "      <td>7.664300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>120500</td>\n",
       "      <td>7.687500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>121000</td>\n",
       "      <td>7.681300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>121500</td>\n",
       "      <td>7.679100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>122000</td>\n",
       "      <td>7.690800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>122500</td>\n",
       "      <td>7.662800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>123000</td>\n",
       "      <td>7.690400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>123500</td>\n",
       "      <td>7.664400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>124000</td>\n",
       "      <td>7.680400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>124500</td>\n",
       "      <td>7.689100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>125000</td>\n",
       "      <td>7.687200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>125500</td>\n",
       "      <td>7.705700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>126000</td>\n",
       "      <td>7.670100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>126500</td>\n",
       "      <td>7.665300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>127000</td>\n",
       "      <td>7.639800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>127500</td>\n",
       "      <td>7.663500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>128000</td>\n",
       "      <td>7.691600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>128500</td>\n",
       "      <td>7.662500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>129000</td>\n",
       "      <td>7.682400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>129500</td>\n",
       "      <td>7.669700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>130000</td>\n",
       "      <td>7.648500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>130500</td>\n",
       "      <td>7.639400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>131000</td>\n",
       "      <td>7.668100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>131500</td>\n",
       "      <td>7.637100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>132000</td>\n",
       "      <td>7.659300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>132500</td>\n",
       "      <td>7.679200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>133000</td>\n",
       "      <td>7.655000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>133500</td>\n",
       "      <td>7.636300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>134000</td>\n",
       "      <td>7.657300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>134500</td>\n",
       "      <td>7.614200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>135000</td>\n",
       "      <td>7.672100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>135500</td>\n",
       "      <td>7.645500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>136000</td>\n",
       "      <td>7.630400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>136500</td>\n",
       "      <td>7.623900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>137000</td>\n",
       "      <td>7.629300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>137500</td>\n",
       "      <td>7.631700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>138000</td>\n",
       "      <td>7.644900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>138500</td>\n",
       "      <td>7.637900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>139000</td>\n",
       "      <td>7.637000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>139500</td>\n",
       "      <td>7.606700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>140000</td>\n",
       "      <td>7.623300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>140500</td>\n",
       "      <td>7.620400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>141000</td>\n",
       "      <td>7.628100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>141500</td>\n",
       "      <td>7.614900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>142000</td>\n",
       "      <td>7.614300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>142500</td>\n",
       "      <td>7.662300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>143000</td>\n",
       "      <td>7.629500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>143500</td>\n",
       "      <td>7.633800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>144000</td>\n",
       "      <td>7.620900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>144500</td>\n",
       "      <td>7.619900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>145000</td>\n",
       "      <td>7.607500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>145500</td>\n",
       "      <td>7.622000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>146000</td>\n",
       "      <td>7.612100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>146500</td>\n",
       "      <td>7.590500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>147000</td>\n",
       "      <td>7.625700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>147500</td>\n",
       "      <td>7.633000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>148000</td>\n",
       "      <td>7.625900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>148500</td>\n",
       "      <td>7.588000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>149000</td>\n",
       "      <td>7.618500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>149500</td>\n",
       "      <td>7.642200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>150000</td>\n",
       "      <td>7.654100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>150500</td>\n",
       "      <td>7.612700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>151000</td>\n",
       "      <td>7.586800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>151500</td>\n",
       "      <td>7.622700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>152000</td>\n",
       "      <td>7.605600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>152500</td>\n",
       "      <td>7.591500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>153000</td>\n",
       "      <td>7.617000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>153500</td>\n",
       "      <td>7.596300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>154000</td>\n",
       "      <td>7.612800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>154500</td>\n",
       "      <td>7.619400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>155000</td>\n",
       "      <td>7.624300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>155500</td>\n",
       "      <td>7.596100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>156000</td>\n",
       "      <td>7.631100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>156500</td>\n",
       "      <td>7.569500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>157000</td>\n",
       "      <td>7.607300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>157500</td>\n",
       "      <td>7.591900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>158000</td>\n",
       "      <td>7.588800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>158500</td>\n",
       "      <td>7.621000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>159000</td>\n",
       "      <td>7.621400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>159500</td>\n",
       "      <td>7.608100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>160000</td>\n",
       "      <td>7.573000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>160500</td>\n",
       "      <td>7.594300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>161000</td>\n",
       "      <td>7.599900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>161500</td>\n",
       "      <td>7.587500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>162000</td>\n",
       "      <td>7.616400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>162500</td>\n",
       "      <td>7.573900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>163000</td>\n",
       "      <td>7.597800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>163500</td>\n",
       "      <td>7.579400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>164000</td>\n",
       "      <td>7.576500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>164500</td>\n",
       "      <td>7.564300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>165000</td>\n",
       "      <td>7.571400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>165500</td>\n",
       "      <td>7.595200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>166000</td>\n",
       "      <td>7.589100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>166500</td>\n",
       "      <td>7.599500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>167000</td>\n",
       "      <td>7.575800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>167500</td>\n",
       "      <td>7.586800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>168000</td>\n",
       "      <td>7.569000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>168500</td>\n",
       "      <td>7.576900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>169000</td>\n",
       "      <td>7.573600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>169500</td>\n",
       "      <td>7.579600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>170000</td>\n",
       "      <td>7.595300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>170500</td>\n",
       "      <td>7.586600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>171000</td>\n",
       "      <td>7.591600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>171500</td>\n",
       "      <td>7.589600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>172000</td>\n",
       "      <td>7.556600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>172500</td>\n",
       "      <td>7.569500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>173000</td>\n",
       "      <td>7.561700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>173500</td>\n",
       "      <td>7.576800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>174000</td>\n",
       "      <td>7.548500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>174500</td>\n",
       "      <td>7.568800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>175000</td>\n",
       "      <td>7.549900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>175500</td>\n",
       "      <td>7.572600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>176000</td>\n",
       "      <td>7.572800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>176500</td>\n",
       "      <td>7.567100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>177000</td>\n",
       "      <td>7.551200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>177500</td>\n",
       "      <td>7.561600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>178000</td>\n",
       "      <td>7.538900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>178500</td>\n",
       "      <td>7.532600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>179000</td>\n",
       "      <td>7.554100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>179500</td>\n",
       "      <td>7.551200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>180000</td>\n",
       "      <td>7.570200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>180500</td>\n",
       "      <td>7.565000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>181000</td>\n",
       "      <td>7.534800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>181500</td>\n",
       "      <td>7.551600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>182000</td>\n",
       "      <td>7.549500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>182500</td>\n",
       "      <td>7.578000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>183000</td>\n",
       "      <td>7.557900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>183500</td>\n",
       "      <td>7.541200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>184000</td>\n",
       "      <td>7.536400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>184500</td>\n",
       "      <td>7.549300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>185000</td>\n",
       "      <td>7.538300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>185500</td>\n",
       "      <td>7.556700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>186000</td>\n",
       "      <td>7.537200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>186500</td>\n",
       "      <td>7.536600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>187000</td>\n",
       "      <td>7.561800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>187500</td>\n",
       "      <td>7.529800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>188000</td>\n",
       "      <td>7.559300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>188500</td>\n",
       "      <td>7.538700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>189000</td>\n",
       "      <td>7.536500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>189500</td>\n",
       "      <td>7.542700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>190000</td>\n",
       "      <td>7.559400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>190500</td>\n",
       "      <td>7.554400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>191000</td>\n",
       "      <td>7.503800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>191500</td>\n",
       "      <td>7.558500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>192000</td>\n",
       "      <td>7.494200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>192500</td>\n",
       "      <td>7.551100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>193000</td>\n",
       "      <td>7.522500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>193500</td>\n",
       "      <td>7.563200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>194000</td>\n",
       "      <td>7.584700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>194500</td>\n",
       "      <td>7.536000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>195000</td>\n",
       "      <td>7.482100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>195500</td>\n",
       "      <td>7.493900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>196000</td>\n",
       "      <td>7.502000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>196500</td>\n",
       "      <td>7.486900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>197000</td>\n",
       "      <td>7.477000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>197500</td>\n",
       "      <td>7.465600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>198000</td>\n",
       "      <td>7.498600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>198500</td>\n",
       "      <td>7.499400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>199000</td>\n",
       "      <td>7.489600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>199500</td>\n",
       "      <td>7.462600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>200000</td>\n",
       "      <td>7.492900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>200500</td>\n",
       "      <td>7.486100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>201000</td>\n",
       "      <td>7.508500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>201500</td>\n",
       "      <td>7.494400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>202000</td>\n",
       "      <td>7.504800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>202500</td>\n",
       "      <td>7.440900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>203000</td>\n",
       "      <td>7.478300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>203500</td>\n",
       "      <td>7.485100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>204000</td>\n",
       "      <td>7.508900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>204500</td>\n",
       "      <td>7.464800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>205000</td>\n",
       "      <td>7.520300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>205500</td>\n",
       "      <td>7.515300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>206000</td>\n",
       "      <td>7.506500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>206500</td>\n",
       "      <td>7.471200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>207000</td>\n",
       "      <td>7.503600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>207500</td>\n",
       "      <td>7.488800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>208000</td>\n",
       "      <td>7.484900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>208500</td>\n",
       "      <td>7.516400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>209000</td>\n",
       "      <td>7.469200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>209500</td>\n",
       "      <td>7.466800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>210000</td>\n",
       "      <td>7.457000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>210500</td>\n",
       "      <td>7.479700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>211000</td>\n",
       "      <td>7.472800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>211500</td>\n",
       "      <td>7.495900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>212000</td>\n",
       "      <td>7.444700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>212500</td>\n",
       "      <td>7.454100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>213000</td>\n",
       "      <td>7.505700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>213500</td>\n",
       "      <td>7.477200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>214000</td>\n",
       "      <td>7.518800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>214500</td>\n",
       "      <td>7.505400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>215000</td>\n",
       "      <td>7.502200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>215500</td>\n",
       "      <td>7.484100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>216000</td>\n",
       "      <td>7.459100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>216500</td>\n",
       "      <td>7.478800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>217000</td>\n",
       "      <td>7.441900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>217500</td>\n",
       "      <td>7.477700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>218000</td>\n",
       "      <td>7.475100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>218500</td>\n",
       "      <td>7.477300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>219000</td>\n",
       "      <td>7.459300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>219500</td>\n",
       "      <td>7.475800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>220000</td>\n",
       "      <td>7.480200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>220500</td>\n",
       "      <td>7.480800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>221000</td>\n",
       "      <td>7.495200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>221500</td>\n",
       "      <td>7.480500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>222000</td>\n",
       "      <td>7.429800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>222500</td>\n",
       "      <td>7.474100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>223000</td>\n",
       "      <td>7.479000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>223500</td>\n",
       "      <td>7.461500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>224000</td>\n",
       "      <td>7.494100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>224500</td>\n",
       "      <td>7.474200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>225000</td>\n",
       "      <td>7.434900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>225500</td>\n",
       "      <td>7.448700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>226000</td>\n",
       "      <td>7.498400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>226500</td>\n",
       "      <td>7.459400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>227000</td>\n",
       "      <td>7.473500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>227500</td>\n",
       "      <td>7.479000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>228000</td>\n",
       "      <td>7.451700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>228500</td>\n",
       "      <td>7.474500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>229000</td>\n",
       "      <td>7.466800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>229500</td>\n",
       "      <td>7.467600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>230000</td>\n",
       "      <td>7.466800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>230500</td>\n",
       "      <td>7.465600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>231000</td>\n",
       "      <td>7.443400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>231500</td>\n",
       "      <td>7.419100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>232000</td>\n",
       "      <td>7.475500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>232500</td>\n",
       "      <td>7.494300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>233000</td>\n",
       "      <td>7.479700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>233500</td>\n",
       "      <td>7.470900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>234000</td>\n",
       "      <td>7.450800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>234500</td>\n",
       "      <td>7.478000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>235000</td>\n",
       "      <td>7.498000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>235500</td>\n",
       "      <td>7.474200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>236000</td>\n",
       "      <td>7.482700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>236500</td>\n",
       "      <td>7.469500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>237000</td>\n",
       "      <td>7.473800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>237500</td>\n",
       "      <td>7.453000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>238000</td>\n",
       "      <td>7.464300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>238500</td>\n",
       "      <td>7.452900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>239000</td>\n",
       "      <td>7.465100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>239500</td>\n",
       "      <td>7.467800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>240000</td>\n",
       "      <td>7.450000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>240500</td>\n",
       "      <td>7.466300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>241000</td>\n",
       "      <td>7.415900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>241500</td>\n",
       "      <td>7.453500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>242000</td>\n",
       "      <td>7.458300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>242500</td>\n",
       "      <td>7.463200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>243000</td>\n",
       "      <td>7.467700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>243500</td>\n",
       "      <td>7.468700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>244000</td>\n",
       "      <td>7.447600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>244500</td>\n",
       "      <td>7.491800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>245000</td>\n",
       "      <td>7.444900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>245500</td>\n",
       "      <td>7.471800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>246000</td>\n",
       "      <td>7.458800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>246500</td>\n",
       "      <td>7.459500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>247000</td>\n",
       "      <td>7.477500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>247500</td>\n",
       "      <td>7.470400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>248000</td>\n",
       "      <td>7.460800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>248500</td>\n",
       "      <td>7.463600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>249000</td>\n",
       "      <td>7.467500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>249500</td>\n",
       "      <td>7.442800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>250000</td>\n",
       "      <td>7.427900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>250500</td>\n",
       "      <td>7.443800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>251000</td>\n",
       "      <td>7.439300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>251500</td>\n",
       "      <td>7.447900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>252000</td>\n",
       "      <td>7.454600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>252500</td>\n",
       "      <td>7.450200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>253000</td>\n",
       "      <td>7.468300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>253500</td>\n",
       "      <td>7.458000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>254000</td>\n",
       "      <td>7.445800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>254500</td>\n",
       "      <td>7.462900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>255000</td>\n",
       "      <td>7.448100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>255500</td>\n",
       "      <td>7.436300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>256000</td>\n",
       "      <td>7.453400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>256500</td>\n",
       "      <td>7.457900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>257000</td>\n",
       "      <td>7.452900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>257500</td>\n",
       "      <td>7.430600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>258000</td>\n",
       "      <td>7.445600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>258500</td>\n",
       "      <td>7.474000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>259000</td>\n",
       "      <td>7.448900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>259500</td>\n",
       "      <td>7.441300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>260000</td>\n",
       "      <td>7.480000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>260500</td>\n",
       "      <td>7.455800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>261000</td>\n",
       "      <td>7.417800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>261500</td>\n",
       "      <td>7.443700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>262000</td>\n",
       "      <td>7.450700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>262500</td>\n",
       "      <td>7.439000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>263000</td>\n",
       "      <td>7.453700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>263500</td>\n",
       "      <td>7.447000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>264000</td>\n",
       "      <td>7.449400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>264500</td>\n",
       "      <td>7.430500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>265000</td>\n",
       "      <td>7.433800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>265500</td>\n",
       "      <td>7.418200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>266000</td>\n",
       "      <td>7.462600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>266500</td>\n",
       "      <td>7.466600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>267000</td>\n",
       "      <td>7.437700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>267500</td>\n",
       "      <td>7.456100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>268000</td>\n",
       "      <td>7.434900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>268500</td>\n",
       "      <td>7.440400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>269000</td>\n",
       "      <td>7.439200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>269500</td>\n",
       "      <td>7.466400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>270000</td>\n",
       "      <td>7.418800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>270500</td>\n",
       "      <td>7.444400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>271000</td>\n",
       "      <td>7.459100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>271500</td>\n",
       "      <td>7.440000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>272000</td>\n",
       "      <td>7.442800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>272500</td>\n",
       "      <td>7.429700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>273000</td>\n",
       "      <td>7.460700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>273500</td>\n",
       "      <td>7.441500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>274000</td>\n",
       "      <td>7.430500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>274500</td>\n",
       "      <td>7.405900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>275000</td>\n",
       "      <td>7.453000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>275500</td>\n",
       "      <td>7.443000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>276000</td>\n",
       "      <td>7.424900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>276500</td>\n",
       "      <td>7.416500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>277000</td>\n",
       "      <td>7.430600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>277500</td>\n",
       "      <td>7.376800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>278000</td>\n",
       "      <td>7.444200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>278500</td>\n",
       "      <td>7.448500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>279000</td>\n",
       "      <td>7.454100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>279500</td>\n",
       "      <td>7.443900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>280000</td>\n",
       "      <td>7.399600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>280500</td>\n",
       "      <td>7.430900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>281000</td>\n",
       "      <td>7.436200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>281500</td>\n",
       "      <td>7.438000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>282000</td>\n",
       "      <td>7.433700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>282500</td>\n",
       "      <td>7.427200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>283000</td>\n",
       "      <td>7.452000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>283500</td>\n",
       "      <td>7.407200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>284000</td>\n",
       "      <td>7.392600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>284500</td>\n",
       "      <td>7.428800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>285000</td>\n",
       "      <td>7.445600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>285500</td>\n",
       "      <td>7.430700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>286000</td>\n",
       "      <td>7.424700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>286500</td>\n",
       "      <td>7.387300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>287000</td>\n",
       "      <td>7.422600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>287500</td>\n",
       "      <td>7.398800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>288000</td>\n",
       "      <td>7.448700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>288500</td>\n",
       "      <td>7.431000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>289000</td>\n",
       "      <td>7.411400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>289500</td>\n",
       "      <td>7.442600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>290000</td>\n",
       "      <td>7.437000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>290500</td>\n",
       "      <td>7.422600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>291000</td>\n",
       "      <td>7.421200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>291500</td>\n",
       "      <td>7.444300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>292000</td>\n",
       "      <td>7.395700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>292500</td>\n",
       "      <td>7.384800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>293000</td>\n",
       "      <td>7.361800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>293500</td>\n",
       "      <td>7.371100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>294000</td>\n",
       "      <td>7.371900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>294500</td>\n",
       "      <td>7.354200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>295000</td>\n",
       "      <td>7.394000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>295500</td>\n",
       "      <td>7.377900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>296000</td>\n",
       "      <td>7.379800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>296500</td>\n",
       "      <td>7.362300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>297000</td>\n",
       "      <td>7.394200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>297500</td>\n",
       "      <td>7.404000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>298000</td>\n",
       "      <td>7.404800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>298500</td>\n",
       "      <td>7.385600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>299000</td>\n",
       "      <td>7.376700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>299500</td>\n",
       "      <td>7.375900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>300000</td>\n",
       "      <td>7.359700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>300500</td>\n",
       "      <td>7.365700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>301000</td>\n",
       "      <td>7.378800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>301500</td>\n",
       "      <td>7.372100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>302000</td>\n",
       "      <td>7.370300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>302500</td>\n",
       "      <td>7.371800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>303000</td>\n",
       "      <td>7.402600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>303500</td>\n",
       "      <td>7.394700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>304000</td>\n",
       "      <td>7.351100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>304500</td>\n",
       "      <td>7.382500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>305000</td>\n",
       "      <td>7.369900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>305500</td>\n",
       "      <td>7.377500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>306000</td>\n",
       "      <td>7.396400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>306500</td>\n",
       "      <td>7.352300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>307000</td>\n",
       "      <td>7.363600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>307500</td>\n",
       "      <td>7.407200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>308000</td>\n",
       "      <td>7.371100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>308500</td>\n",
       "      <td>7.391600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>309000</td>\n",
       "      <td>7.364900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>309500</td>\n",
       "      <td>7.346900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>310000</td>\n",
       "      <td>7.365800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>310500</td>\n",
       "      <td>7.358400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>311000</td>\n",
       "      <td>7.342700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>311500</td>\n",
       "      <td>7.373300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>312000</td>\n",
       "      <td>7.389700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>312500</td>\n",
       "      <td>7.386800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>313000</td>\n",
       "      <td>7.367400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>313500</td>\n",
       "      <td>7.379600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>314000</td>\n",
       "      <td>7.386600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>314500</td>\n",
       "      <td>7.361100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>315000</td>\n",
       "      <td>7.349800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>315500</td>\n",
       "      <td>7.355500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>316000</td>\n",
       "      <td>7.368300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>316500</td>\n",
       "      <td>7.371800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>317000</td>\n",
       "      <td>7.398400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>317500</td>\n",
       "      <td>7.391200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>318000</td>\n",
       "      <td>7.354500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>318500</td>\n",
       "      <td>7.384600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>319000</td>\n",
       "      <td>7.378200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>319500</td>\n",
       "      <td>7.349400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>320000</td>\n",
       "      <td>7.370700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>320500</td>\n",
       "      <td>7.378000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>321000</td>\n",
       "      <td>7.373400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>321500</td>\n",
       "      <td>7.362100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>322000</td>\n",
       "      <td>7.357000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>322500</td>\n",
       "      <td>7.372300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>323000</td>\n",
       "      <td>7.384200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>323500</td>\n",
       "      <td>7.345900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>324000</td>\n",
       "      <td>7.382600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>324500</td>\n",
       "      <td>7.335000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>325000</td>\n",
       "      <td>7.395500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>325500</td>\n",
       "      <td>7.383900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>326000</td>\n",
       "      <td>7.405500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>326500</td>\n",
       "      <td>7.370200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>327000</td>\n",
       "      <td>7.318200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>327500</td>\n",
       "      <td>7.343800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>328000</td>\n",
       "      <td>7.393900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>328500</td>\n",
       "      <td>7.367800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>329000</td>\n",
       "      <td>7.362300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>329500</td>\n",
       "      <td>7.352900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>330000</td>\n",
       "      <td>7.364300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>330500</td>\n",
       "      <td>7.375200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>331000</td>\n",
       "      <td>7.329600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>331500</td>\n",
       "      <td>7.357100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>332000</td>\n",
       "      <td>7.324200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>332500</td>\n",
       "      <td>7.409000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>333000</td>\n",
       "      <td>7.360500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>333500</td>\n",
       "      <td>7.370500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>334000</td>\n",
       "      <td>7.366900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>334500</td>\n",
       "      <td>7.296700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>335000</td>\n",
       "      <td>7.364400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>335500</td>\n",
       "      <td>7.367300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>336000</td>\n",
       "      <td>7.356600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>336500</td>\n",
       "      <td>7.373300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>337000</td>\n",
       "      <td>7.362800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>337500</td>\n",
       "      <td>7.381600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>338000</td>\n",
       "      <td>7.336100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>338500</td>\n",
       "      <td>7.315100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>339000</td>\n",
       "      <td>7.388100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>339500</td>\n",
       "      <td>7.380700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>340000</td>\n",
       "      <td>7.373300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>340500</td>\n",
       "      <td>7.352000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>341000</td>\n",
       "      <td>7.355600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>341500</td>\n",
       "      <td>7.353600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>342000</td>\n",
       "      <td>7.357100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>342500</td>\n",
       "      <td>7.365000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>343000</td>\n",
       "      <td>7.372100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>343500</td>\n",
       "      <td>7.349600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>344000</td>\n",
       "      <td>7.367600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>344500</td>\n",
       "      <td>7.347500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>345000</td>\n",
       "      <td>7.388600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>345500</td>\n",
       "      <td>7.383100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>346000</td>\n",
       "      <td>7.333700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>346500</td>\n",
       "      <td>7.370500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>347000</td>\n",
       "      <td>7.369800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>347500</td>\n",
       "      <td>7.367900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>348000</td>\n",
       "      <td>7.310000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>348500</td>\n",
       "      <td>7.348000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>349000</td>\n",
       "      <td>7.335900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>349500</td>\n",
       "      <td>7.343700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>350000</td>\n",
       "      <td>7.355400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>350500</td>\n",
       "      <td>7.334200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>351000</td>\n",
       "      <td>7.380400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>351500</td>\n",
       "      <td>7.373300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>352000</td>\n",
       "      <td>7.375800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>352500</td>\n",
       "      <td>7.346600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>353000</td>\n",
       "      <td>7.353800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>353500</td>\n",
       "      <td>7.380800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>354000</td>\n",
       "      <td>7.344900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>354500</td>\n",
       "      <td>7.358400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>355000</td>\n",
       "      <td>7.342400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>355500</td>\n",
       "      <td>7.380900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>356000</td>\n",
       "      <td>7.367100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>356500</td>\n",
       "      <td>7.329800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>357000</td>\n",
       "      <td>7.324900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>357500</td>\n",
       "      <td>7.394300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>358000</td>\n",
       "      <td>7.345700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>358500</td>\n",
       "      <td>7.366600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>359000</td>\n",
       "      <td>7.374900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>359500</td>\n",
       "      <td>7.334800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>360000</td>\n",
       "      <td>7.364900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>360500</td>\n",
       "      <td>7.318900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>361000</td>\n",
       "      <td>7.333400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>361500</td>\n",
       "      <td>7.341900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>362000</td>\n",
       "      <td>7.326900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>362500</td>\n",
       "      <td>7.343900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>363000</td>\n",
       "      <td>7.330400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>363500</td>\n",
       "      <td>7.353100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>364000</td>\n",
       "      <td>7.354600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>364500</td>\n",
       "      <td>7.362300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>365000</td>\n",
       "      <td>7.332600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>365500</td>\n",
       "      <td>7.352500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>366000</td>\n",
       "      <td>7.334700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>366500</td>\n",
       "      <td>7.357300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>367000</td>\n",
       "      <td>7.342800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>367500</td>\n",
       "      <td>7.346300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>368000</td>\n",
       "      <td>7.355300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>368500</td>\n",
       "      <td>7.327800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>369000</td>\n",
       "      <td>7.340000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>369500</td>\n",
       "      <td>7.344700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>370000</td>\n",
       "      <td>7.360100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>370500</td>\n",
       "      <td>7.327300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>371000</td>\n",
       "      <td>7.332200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>371500</td>\n",
       "      <td>7.369600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>372000</td>\n",
       "      <td>7.342000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>372500</td>\n",
       "      <td>7.359500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>373000</td>\n",
       "      <td>7.330100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>373500</td>\n",
       "      <td>7.377900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>374000</td>\n",
       "      <td>7.348200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>374500</td>\n",
       "      <td>7.365700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>375000</td>\n",
       "      <td>7.368400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>375500</td>\n",
       "      <td>7.348900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>376000</td>\n",
       "      <td>7.339300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>376500</td>\n",
       "      <td>7.313200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>377000</td>\n",
       "      <td>7.342100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>377500</td>\n",
       "      <td>7.368000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>378000</td>\n",
       "      <td>7.334700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>378500</td>\n",
       "      <td>7.370500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>379000</td>\n",
       "      <td>7.295100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>379500</td>\n",
       "      <td>7.304200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>380000</td>\n",
       "      <td>7.379100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>380500</td>\n",
       "      <td>7.336400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>381000</td>\n",
       "      <td>7.342900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>381500</td>\n",
       "      <td>7.346600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>382000</td>\n",
       "      <td>7.328700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>382500</td>\n",
       "      <td>7.348100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>383000</td>\n",
       "      <td>7.326700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>383500</td>\n",
       "      <td>7.331200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>384000</td>\n",
       "      <td>7.333200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>384500</td>\n",
       "      <td>7.345300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>385000</td>\n",
       "      <td>7.331600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>385500</td>\n",
       "      <td>7.351200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>386000</td>\n",
       "      <td>7.307700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>386500</td>\n",
       "      <td>7.327300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>387000</td>\n",
       "      <td>7.347600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>387500</td>\n",
       "      <td>7.388400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>388000</td>\n",
       "      <td>7.354200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>388500</td>\n",
       "      <td>7.311600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>389000</td>\n",
       "      <td>7.300000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>389500</td>\n",
       "      <td>7.270900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>390000</td>\n",
       "      <td>7.312300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>390500</td>\n",
       "      <td>7.272900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>391000</td>\n",
       "      <td>7.299000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>391500</td>\n",
       "      <td>7.303600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>392000</td>\n",
       "      <td>7.264500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>392500</td>\n",
       "      <td>7.321000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>393000</td>\n",
       "      <td>7.302300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>393500</td>\n",
       "      <td>7.328100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>394000</td>\n",
       "      <td>7.322000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>394500</td>\n",
       "      <td>7.301300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>395000</td>\n",
       "      <td>7.325700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>395500</td>\n",
       "      <td>7.258900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>396000</td>\n",
       "      <td>7.320800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>396500</td>\n",
       "      <td>7.340100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>397000</td>\n",
       "      <td>7.292200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>397500</td>\n",
       "      <td>7.320700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>398000</td>\n",
       "      <td>7.288200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>398500</td>\n",
       "      <td>7.293500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>399000</td>\n",
       "      <td>7.293300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>399500</td>\n",
       "      <td>7.270400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400000</td>\n",
       "      <td>7.309700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>400500</td>\n",
       "      <td>7.301200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>401000</td>\n",
       "      <td>7.311000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>401500</td>\n",
       "      <td>7.292200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>402000</td>\n",
       "      <td>7.328500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>402500</td>\n",
       "      <td>7.305800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>403000</td>\n",
       "      <td>7.271900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>403500</td>\n",
       "      <td>7.295200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>404000</td>\n",
       "      <td>7.297300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>404500</td>\n",
       "      <td>7.304700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>405000</td>\n",
       "      <td>7.304800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>405500</td>\n",
       "      <td>7.342400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>406000</td>\n",
       "      <td>7.306100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>406500</td>\n",
       "      <td>7.307100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>407000</td>\n",
       "      <td>7.285800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>407500</td>\n",
       "      <td>7.295400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>408000</td>\n",
       "      <td>7.300300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>408500</td>\n",
       "      <td>7.284900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>409000</td>\n",
       "      <td>7.314000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>409500</td>\n",
       "      <td>7.315700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>410000</td>\n",
       "      <td>7.332800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>410500</td>\n",
       "      <td>7.313300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>411000</td>\n",
       "      <td>7.290600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>411500</td>\n",
       "      <td>7.293200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>412000</td>\n",
       "      <td>7.292800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>412500</td>\n",
       "      <td>7.287900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>413000</td>\n",
       "      <td>7.310700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>413500</td>\n",
       "      <td>7.290400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>414000</td>\n",
       "      <td>7.299000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>414500</td>\n",
       "      <td>7.317200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>415000</td>\n",
       "      <td>7.303900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>415500</td>\n",
       "      <td>7.297000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>416000</td>\n",
       "      <td>7.284000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>416500</td>\n",
       "      <td>7.314400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>417000</td>\n",
       "      <td>7.307300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>417500</td>\n",
       "      <td>7.310300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>418000</td>\n",
       "      <td>7.292100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>418500</td>\n",
       "      <td>7.308900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>419000</td>\n",
       "      <td>7.327000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>419500</td>\n",
       "      <td>7.291600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>420000</td>\n",
       "      <td>7.310000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>420500</td>\n",
       "      <td>7.325800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>421000</td>\n",
       "      <td>7.298000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>421500</td>\n",
       "      <td>7.285100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>422000</td>\n",
       "      <td>7.292100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>422500</td>\n",
       "      <td>7.295500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>423000</td>\n",
       "      <td>7.284100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>423500</td>\n",
       "      <td>7.306100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>424000</td>\n",
       "      <td>7.284200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>424500</td>\n",
       "      <td>7.301100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>425000</td>\n",
       "      <td>7.275300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>425500</td>\n",
       "      <td>7.255800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>426000</td>\n",
       "      <td>7.298200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>426500</td>\n",
       "      <td>7.294300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>427000</td>\n",
       "      <td>7.276600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>427500</td>\n",
       "      <td>7.284100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>428000</td>\n",
       "      <td>7.271200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>428500</td>\n",
       "      <td>7.299700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>429000</td>\n",
       "      <td>7.295900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>429500</td>\n",
       "      <td>7.324600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>430000</td>\n",
       "      <td>7.283800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>430500</td>\n",
       "      <td>7.303100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>431000</td>\n",
       "      <td>7.267400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>431500</td>\n",
       "      <td>7.328000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>432000</td>\n",
       "      <td>7.297100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>432500</td>\n",
       "      <td>7.301900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>433000</td>\n",
       "      <td>7.305000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>433500</td>\n",
       "      <td>7.302500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>434000</td>\n",
       "      <td>7.288600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>434500</td>\n",
       "      <td>7.289800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>435000</td>\n",
       "      <td>7.332400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>435500</td>\n",
       "      <td>7.280300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>436000</td>\n",
       "      <td>7.290900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>436500</td>\n",
       "      <td>7.274600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>437000</td>\n",
       "      <td>7.294600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>437500</td>\n",
       "      <td>7.321900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>438000</td>\n",
       "      <td>7.277900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>438500</td>\n",
       "      <td>7.297200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>439000</td>\n",
       "      <td>7.298400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>439500</td>\n",
       "      <td>7.256400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>440000</td>\n",
       "      <td>7.325100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>440500</td>\n",
       "      <td>7.264500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>441000</td>\n",
       "      <td>7.315200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>441500</td>\n",
       "      <td>7.326400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>442000</td>\n",
       "      <td>7.304200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>442500</td>\n",
       "      <td>7.288700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>443000</td>\n",
       "      <td>7.305800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>443500</td>\n",
       "      <td>7.281300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>444000</td>\n",
       "      <td>7.278600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>444500</td>\n",
       "      <td>7.272900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>445000</td>\n",
       "      <td>7.274000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>445500</td>\n",
       "      <td>7.323800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>446000</td>\n",
       "      <td>7.333400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>446500</td>\n",
       "      <td>7.307800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>447000</td>\n",
       "      <td>7.309000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>447500</td>\n",
       "      <td>7.305300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>448000</td>\n",
       "      <td>7.259000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>448500</td>\n",
       "      <td>7.320700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>449000</td>\n",
       "      <td>7.271000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>449500</td>\n",
       "      <td>7.281700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>450000</td>\n",
       "      <td>7.266700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>450500</td>\n",
       "      <td>7.279800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>451000</td>\n",
       "      <td>7.266500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>451500</td>\n",
       "      <td>7.310900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>452000</td>\n",
       "      <td>7.304600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>452500</td>\n",
       "      <td>7.262700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>453000</td>\n",
       "      <td>7.283700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>453500</td>\n",
       "      <td>7.294300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>454000</td>\n",
       "      <td>7.283000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>454500</td>\n",
       "      <td>7.296900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>455000</td>\n",
       "      <td>7.272700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>455500</td>\n",
       "      <td>7.280400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>456000</td>\n",
       "      <td>7.347300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>456500</td>\n",
       "      <td>7.312800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>457000</td>\n",
       "      <td>7.278400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>457500</td>\n",
       "      <td>7.319100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>458000</td>\n",
       "      <td>7.320700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>458500</td>\n",
       "      <td>7.287400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>459000</td>\n",
       "      <td>7.268900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>459500</td>\n",
       "      <td>7.299200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>460000</td>\n",
       "      <td>7.314500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>460500</td>\n",
       "      <td>7.294700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>461000</td>\n",
       "      <td>7.315700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>461500</td>\n",
       "      <td>7.317600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>462000</td>\n",
       "      <td>7.307100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>462500</td>\n",
       "      <td>7.263600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>463000</td>\n",
       "      <td>7.278600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>463500</td>\n",
       "      <td>7.298900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>464000</td>\n",
       "      <td>7.293600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>464500</td>\n",
       "      <td>7.283700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>465000</td>\n",
       "      <td>7.296900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>465500</td>\n",
       "      <td>7.275100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>466000</td>\n",
       "      <td>7.257100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>466500</td>\n",
       "      <td>7.298500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>467000</td>\n",
       "      <td>7.284000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>467500</td>\n",
       "      <td>7.293800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>468000</td>\n",
       "      <td>7.261200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>468500</td>\n",
       "      <td>7.306500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>469000</td>\n",
       "      <td>7.293900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>469500</td>\n",
       "      <td>7.276000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>470000</td>\n",
       "      <td>7.279200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>470500</td>\n",
       "      <td>7.312000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>471000</td>\n",
       "      <td>7.250800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>471500</td>\n",
       "      <td>7.325500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>472000</td>\n",
       "      <td>7.275200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>472500</td>\n",
       "      <td>7.284100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>473000</td>\n",
       "      <td>7.284600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>473500</td>\n",
       "      <td>7.305900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>474000</td>\n",
       "      <td>7.263100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>474500</td>\n",
       "      <td>7.297000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>475000</td>\n",
       "      <td>7.313300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>475500</td>\n",
       "      <td>7.279000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>476000</td>\n",
       "      <td>7.266500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>476500</td>\n",
       "      <td>7.283600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>477000</td>\n",
       "      <td>7.318700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>477500</td>\n",
       "      <td>7.277700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>478000</td>\n",
       "      <td>7.297300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>478500</td>\n",
       "      <td>7.288900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>479000</td>\n",
       "      <td>7.270500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>479500</td>\n",
       "      <td>7.286100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>480000</td>\n",
       "      <td>7.297900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>480500</td>\n",
       "      <td>7.289100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>481000</td>\n",
       "      <td>7.293100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>481500</td>\n",
       "      <td>7.236500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>482000</td>\n",
       "      <td>7.311100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>482500</td>\n",
       "      <td>7.307100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>483000</td>\n",
       "      <td>7.315700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>483500</td>\n",
       "      <td>7.304800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>484000</td>\n",
       "      <td>7.307100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>484500</td>\n",
       "      <td>7.275100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>485000</td>\n",
       "      <td>7.287300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>485500</td>\n",
       "      <td>7.277000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "trainer.train()\n",
    "trainer.save_model(\"dna_gpt2_v0\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "be3178d2-f8d4-4739-87b5-cf397dcb15fa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='13495' max='13495' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [13495/13495 03:30]\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Perplexity: 1509.44\n"
     ]
    }
   ],
   "source": [
    "import math\n",
    "eval_results = trainer.evaluate()\n",
    "print(f\"Perplexity: {math.exp(eval_results['eval_loss']):.2f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "11a49d8e-2e63-486c-bf6d-79af3ef64e98",
   "metadata": {},
   "outputs": [],
   "source": [
    "#upload model\n",
    "#model.push_to_hub(\"dna_gpt2_v0\", organization=\"dnagpt\", use_auth_token=\"hf_*******\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdcab501-7c80-4da6-a2a6-d8ff3e057966",
   "metadata": {},
   "source": [
    "训练完成后，我们就可以直接使用这个模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "97cabc49-9ca2-4c13-a39f-e4a1bdff2304",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GPT2Model(\n",
       "  (wte): Embedding(30000, 768)\n",
       "  (wpe): Embedding(1024, 768)\n",
       "  (drop): Dropout(p=0.1, inplace=False)\n",
       "  (h): ModuleList(\n",
       "    (0-11): 12 x GPT2Block(\n",
       "      (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       "      (attn): GPT2SdpaAttention(\n",
       "        (c_attn): Conv1D(nf=2304, nx=768)\n",
       "        (c_proj): Conv1D(nf=768, nx=768)\n",
       "        (attn_dropout): Dropout(p=0.1, inplace=False)\n",
       "        (resid_dropout): Dropout(p=0.1, inplace=False)\n",
       "      )\n",
       "      (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       "      (mlp): GPT2MLP(\n",
       "        (c_fc): Conv1D(nf=3072, nx=768)\n",
       "        (c_proj): Conv1D(nf=768, nx=3072)\n",
       "        (act): NewGELUActivation()\n",
       "        (dropout): Dropout(p=0.1, inplace=False)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (ln_f): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
       ")"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers import AutoTokenizer, AutoModel\n",
    "import torch\n",
    "model = AutoModel.from_pretrained('dna_gpt2_v0')\n",
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "feda9c88-3b18-435f-bf40-ec42f5c325d8",
   "metadata": {},
   "source": [
    "这里可以更清楚的看到gpt2的网络结构。 GPT-2 模型结构展示了其内部组件和层次。\n",
    "\n",
    "### 总体架构\n",
    "\n",
    "GPT-2 是基于 Transformer 的语言模型，采用了编码器-解码器架构中的解码器部分（即仅包含解码器）。它通过堆叠多个 Transformer 解码器层来处理输入序列，并生成输出序列。\n",
    "\n",
    "### 详细解析\n",
    "\n",
    "#### 1. **嵌入层 (Embedding Layers)**\n",
    "\n",
    "```python\n",
    "(wte): Embedding(30000, 768)\n",
    "(wpe): Embedding(1024, 768)\n",
    "```\n",
    "\n",
    "- **`wte` (Word Token Embeddings)**：这是一个嵌入层，用于将词汇表中的 token 映射到 768 维的向量空间。`30000` 表示词汇表大小，即模型可以识别的 token 数量。\n",
    "  \n",
    "- **`wpe` (Positional Embeddings)**：这是位置嵌入层，用于捕捉序列中每个位置的信息。`1024` 表示模型支持的最大序列长度，`768` 表示每个位置嵌入的维度。\n",
    "\n",
    "#### 2. **Dropout 层**\n",
    "\n",
    "```python\n",
    "(drop): Dropout(p=0.1, inplace=False)\n",
    "```\n",
    "\n",
    "- **`drop`**：这是一个 Dropout 层，用于防止过拟合。在训练过程中，它会随机丢弃一部分神经元，比例为 `p=0.1`（即 10%）。`inplace=False` 表示不会原地修改输入张量。\n",
    "\n",
    "#### 3. **Transformer 块 (Transformer Blocks)**\n",
    "\n",
    "```python\n",
    "(h): ModuleList(\n",
    "    (0-11): 12 x GPT2Block(\n",
    "      ...\n",
    "    )\n",
    ")\n",
    "```\n",
    "\n",
    "- **`h`**：这是一个 `ModuleList`，包含了 12 个 `GPT2Block`，每个块代表一个 Transformer 解码器层。这些层是顺序堆叠的，每一层都会对输入进行进一步的变换和特征提取。\n",
    "\n",
    "##### 每个 `GPT2Block` 的结构如下：\n",
    "\n",
    "- **Layer Normalization (`ln_1` 和 `ln_2`)**：\n",
    "\n",
    "  ```python\n",
    "  (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
    "  (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
    "  ```\n",
    "\n",
    "  - **`ln_1` 和 `ln_2`**：这两个层归一化层分别位于自注意力机制之前和前馈神经网络之前。它们确保了每一层的输入具有相同的分布，从而加速训练并提高稳定性。\n",
    "\n",
    "- **自注意力机制 (`attn`)**：\n",
    "\n",
    "  ```python\n",
    "  (attn): GPT2SdpaAttention(\n",
    "    (c_attn): Conv1D(nf=2304, nx=768)\n",
    "    (c_proj): Conv1D(nf=768, nx=768)\n",
    "    (attn_dropout): Dropout(p=0.1, inplace=False)\n",
    "    (resid_dropout): Dropout(p=0.1, inplace=False)\n",
    "  )\n",
    "  ```\n",
    "\n",
    "  - **`c_attn`**：这是一个线性投影层，将输入映射到查询（Q）、键（K）和值（V）的空间，总维度为 `2304`（因为 Q、K、V 各占 `768` 维）。\n",
    "  - **`c_proj`**：这是另一个线性投影层，将自注意力机制的结果重新映射回原始维度 `768`。\n",
    "  - **`attn_dropout` 和 `resid_dropout`**：这两个 Dropout 层分别应用于注意力得分和残差连接，以防止过拟合。\n",
    "\n",
    "- **前馈神经网络 (`mlp`)**：\n",
    "\n",
    "  ```python\n",
    "  (mlp): GPT2MLP(\n",
    "    (c_fc): Conv1D(nf=3072, nx=768)\n",
    "    (c_proj): Conv1D(nf=768, nx=3072)\n",
    "    (act): NewGELUActivation()\n",
    "    (dropout): Dropout(p=0.1, inplace=False)\n",
    "  )\n",
    "  ```\n",
    "\n",
    "  - **`c_fc`**：这是一个线性投影层，将输入映射到更大的维度 `3072`，以增加表达能力。\n",
    "  - **`c_proj`**：这是另一个线性投影层，将结果重新映射回原始维度 `768`。\n",
    "  - **`act`**：激活函数，使用的是改进版的 GELU（Gaussian Error Linear Unit），有助于非线性变换。\n",
    "  - **`dropout`**：用于防止过拟合的 Dropout 层。\n",
    "\n",
    "#### 4. **最终的 Layer Normalization**\n",
    "\n",
    "```python\n",
    "(ln_f): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
    "```\n",
    "\n",
    "- **`ln_f`**：这是最后一个 Layer Normalization 层，应用于所有 Transformer 块之后的输出。它确保了最终输出的稳定性和一致性。\n",
    "\n",
    "### 工作流程\n",
    "\n",
    "1. **输入处理**：\n",
    "   - 输入 token 通过 `wte` 转换为词嵌入。\n",
    "   - 输入位置通过 `wpe` 转换为位置嵌入。\n",
    "   - 词嵌入和位置嵌入相加，然后通过 `drop` 进行 Dropout。\n",
    "\n",
    "2. **多层 Transformer 解码器**：\n",
    "   - 输入经过 12 层 `GPT2Block`，每层包括自注意力机制和前馈神经网络。\n",
    "   - 每层之间有残差连接（Residual Connection），并且在每个子模块前后都有 Layer Normalization。\n",
    "\n",
    "3. **最终输出**：\n",
    "   - 最终输出通过 `ln_f` 进行归一化，得到模型的隐状态表示。\n",
    "\n",
    "\n",
    "当然，我们也可以使用Netron等工具，查看网络结构\n",
    "\n",
    "<img src=\"img/gpt2-netron.png\" width=\"600px\" />\n",
    "\n",
    "\n",
    "当然，还有一些更酷炫的：\n",
    "https://bbycroft.net/llm\n",
    "\n",
    "<img src=\"img/llm-visual.png\" width=\"600px\" />"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "4ecf34d5-b090-449d-9aa5-ba90cabf83ea",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'input_ids': tensor([[    1,   191,    29,   753,  1241,  2104, 12297,   357,    85,  4395,\n",
      "         26392,    16]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]])}\n",
      "['last_hidden_state', 'past_key_values']\n"
     ]
    }
   ],
   "source": [
    "tokenizer = AutoTokenizer.from_pretrained('dna_bpe_dict')\n",
    "tokenizer.tokenize(\"GAGCACATTCGCCTGCGTGCGCACTCACACACACGTTCAAAAAGAGTCCATTCGATTCTGGCAGTAG\")\n",
    "\n",
    "dna = \"ACGTAGCATCGGATCTATCTATCGACACTTGGTTATCGATCTACGAGCATCTCGTTAGC\"\n",
    "inputs = tokenizer(dna, return_tensors = 'pt')\n",
    "print(inputs)\n",
    "\n",
    "outputs = model(inputs[\"input_ids\"])\n",
    "print(list(outputs))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "58e0c377-ab25-4265-8a44-e2ef4fa5bffe",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('dna_gpt2_v0/tokenizer_config.json',\n",
       " 'dna_gpt2_v0/special_tokens_map.json',\n",
       " 'dna_gpt2_v0/vocab.json',\n",
       " 'dna_gpt2_v0/merges.txt',\n",
       " 'dna_gpt2_v0/added_tokens.json',\n",
       " 'dna_gpt2_v0/tokenizer.json')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#为了使用方便，我们一般还会把分词器也存到模型目录下\n",
    "tokenizer.save_pretrained(\"dna_gpt2_v0\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cbf27648-d758-45ac-8665-e3466cd2fb65",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}