引入 mdtex2html 更多拓展完善markdown渲染显示, resolve #134, resolve #193 (#135)

* 增加代码高亮功能

* 引入了Pygments库用以高亮,但我不确定是不是真的必要;
* 可以从此处挑选喜欢的样式告诉我: https://pygments.org/styles/ ,目前选用了monokai.
* 是否可以完全剔除parse_text有待商榷.

* 增加表格markdown渲染

* 增加基本表格CSS

* 调整parse_text函数

* refractor: 重写了parse_text

但是我真的好久没写代码了，不清楚这样有没有问题。

update utils

* 根据新的对话框样式调整了代码框样式

* 重写parse_text

现在是挤压连续多个`\n`为单个`\n`

* 将css代码从`presets.py`中分离出来，方便修改

* 再重写 parse_text

现在是真的只替换代码块中的多余空行了。。。
这下总可以了吧QAQ

* 修复实时代码块

---------

Co-authored-by: Chuan Hu <[email protected]>
Co-authored-by: mzlegion <[email protected]>

ChuanhuChatbot.py

@@ -50,6 +50,9 @@ else:
 
 gr.Chatbot.postprocess = postprocess
 
+with open("custom.css", "r") as f:
+    customCSS = f.read()
+
 with gr.Blocks(css=customCSS) as demo:
     history = gr.State([])
     token_count = gr.State([])

custom.css

@@ -0,0 +1,133 @@
+#status_display {
+    display: flex;
+    min-height: 2.5em;
+    align-items: flex-end;
+    justify-content: flex-end;
+}
+#status_display p {
+    font-size: .85em;
+    font-family: monospace;
+    color: var(--text-color-subdued) !important;
+}
+[class *= "message"] {
+    border-radius: var(--radius-xl) !important;
+    border: none;
+    padding: var(--spacing-xl) !important;
+    font-size: var(--text-md) !important;
+    line-height: var(--line-md) !important;
+}
+[data-testid = "bot"] {
+    max-width: 85%;
+    border-bottom-left-radius: 0 !important;
+}
+[data-testid = "user"] {
+    max-width: 85%;
+    width: auto !important;
+    border-bottom-right-radius: 0 !important;
+}
+table {
+    margin: 1em 0;
+    border-collapse: collapse;
+    empty-cells: show;
+}
+td,th {
+    border: 1.2px solid var(--color-border-primary) !important;
+    padding: 0.2em;
+}
+thead {
+    background-color: rgba(175,184,193,0.2);
+}
+thead th {
+    padding: .5em .2em;
+}
+code {
+    display: inline;
+    white-space: break-spaces;
+    border-radius: 6px;
+    margin: 0 2px 0 2px;
+    padding: .2em .4em .1em .4em;
+    background-color: rgba(175,184,193,0.2);
+}
+pre code {
+    display: block;
+    white-space: pre;
+    background-color: hsla(0, 0%, 0%, 80%)!important;
+    border-radius: 10px;
+    padding: 1rem 1.2rem 1rem;
+    margin: 1.2em 2em 1.2em 0.5em;
+    color: #FFF;
+    box-shadow: 6px 6px 16px hsla(0, 0%, 0%, 0.2);
+}
+.codehilite .hll { background-color: #49483e }
+.codehilite .c { color: #75715e } /* Comment */
+.codehilite .err { color: #960050; background-color: #1e0010 } /* Error */
+.codehilite .k { color: #66d9ef } /* Keyword */
+.codehilite .l { color: #ae81ff } /* Literal */
+.codehilite .n { color: #f8f8f2 } /* Name */
+.codehilite .o { color: #f92672 } /* Operator */
+.codehilite .p { color: #f8f8f2 } /* Punctuation */
+.codehilite .ch { color: #75715e } /* Comment.Hashbang */
+.codehilite .cm { color: #75715e } /* Comment.Multiline */
+.codehilite .cp { color: #75715e } /* Comment.Preproc */
+.codehilite .cpf { color: #75715e } /* Comment.PreprocFile */
+.codehilite .c1 { color: #75715e } /* Comment.Single */
+.codehilite .cs { color: #75715e } /* Comment.Special */
+.codehilite .gd { color: #f92672 } /* Generic.Deleted */
+.codehilite .ge { font-style: italic } /* Generic.Emph */
+.codehilite .gi { color: #a6e22e } /* Generic.Inserted */
+.codehilite .gs { font-weight: bold } /* Generic.Strong */
+.codehilite .gu { color: #75715e } /* Generic.Subheading */
+.codehilite .kc { color: #66d9ef } /* Keyword.Constant */
+.codehilite .kd { color: #66d9ef } /* Keyword.Declaration */
+.codehilite .kn { color: #f92672 } /* Keyword.Namespace */
+.codehilite .kp { color: #66d9ef } /* Keyword.Pseudo */
+.codehilite .kr { color: #66d9ef } /* Keyword.Reserved */
+.codehilite .kt { color: #66d9ef } /* Keyword.Type */
+.codehilite .ld { color: #e6db74 } /* Literal.Date */
+.codehilite .m { color: #ae81ff } /* Literal.Number */
+.codehilite .s { color: #e6db74 } /* Literal.String */
+.codehilite .na { color: #a6e22e } /* Name.Attribute */
+.codehilite .nb { color: #f8f8f2 } /* Name.Builtin */
+.codehilite .nc { color: #a6e22e } /* Name.Class */
+.codehilite .no { color: #66d9ef } /* Name.Constant */
+.codehilite .nd { color: #a6e22e } /* Name.Decorator */
+.codehilite .ni { color: #f8f8f2 } /* Name.Entity */
+.codehilite .ne { color: #a6e22e } /* Name.Exception */
+.codehilite .nf { color: #a6e22e } /* Name.Function */
+.codehilite .nl { color: #f8f8f2 } /* Name.Label */
+.codehilite .nn { color: #f8f8f2 } /* Name.Namespace */
+.codehilite .nx { color: #a6e22e } /* Name.Other */
+.codehilite .py { color: #f8f8f2 } /* Name.Property */
+.codehilite .nt { color: #f92672 } /* Name.Tag */
+.codehilite .nv { color: #f8f8f2 } /* Name.Variable */
+.codehilite .ow { color: #f92672 } /* Operator.Word */
+.codehilite .w { color: #f8f8f2 } /* Text.Whitespace */
+.codehilite .mb { color: #ae81ff } /* Literal.Number.Bin */
+.codehilite .mf { color: #ae81ff } /* Literal.Number.Float */
+.codehilite .mh { color: #ae81ff } /* Literal.Number.Hex */
+.codehilite .mi { color: #ae81ff } /* Literal.Number.Integer */
+.codehilite .mo { color: #ae81ff } /* Literal.Number.Oct */
+.codehilite .sa { color: #e6db74 } /* Literal.String.Affix */
+.codehilite .sb { color: #e6db74 } /* Literal.String.Backtick */
+.codehilite .sc { color: #e6db74 } /* Literal.String.Char */
+.codehilite .dl { color: #e6db74 } /* Literal.String.Delimiter */
+.codehilite .sd { color: #e6db74 } /* Literal.String.Doc */
+.codehilite .s2 { color: #e6db74 } /* Literal.String.Double */
+.codehilite .se { color: #ae81ff } /* Literal.String.Escape */
+.codehilite .sh { color: #e6db74 } /* Literal.String.Heredoc */
+.codehilite .si { color: #e6db74 } /* Literal.String.Interpol */
+.codehilite .sx { color: #e6db74 } /* Literal.String.Other */
+.codehilite .sr { color: #e6db74 } /* Literal.String.Regex */
+.codehilite .s1 { color: #e6db74 } /* Literal.String.Single */
+.codehilite .ss { color: #e6db74 } /* Literal.String.Symbol */
+.codehilite .bp { color: #f8f8f2 } /* Name.Builtin.Pseudo */
+.codehilite .fm { color: #a6e22e } /* Name.Function.Magic */
+.codehilite .vc { color: #f8f8f2 } /* Name.Variable.Class */
+.codehilite .vg { color: #f8f8f2 } /* Name.Variable.Global */
+.codehilite .vi { color: #f8f8f2 } /* Name.Variable.Instance */
+.codehilite .vm { color: #f8f8f2 } /* Name.Variable.Magic */
+.codehilite .il { color: #ae81ff } /* Literal.Number.Integer.Long */
+
+* {
+    transition: all 0.6s;
+}

presets.py

@@ -1,6 +1,7 @@
 # -*- coding:utf-8 -*-
 title = """<h1 align="left" style="min-width:200px; margin-top:0;">川虎ChatGPT 🚀</h1>"""
-description = """<div align="center" style="margin:16px 0">
+description = """\
+<div align="center" style="margin:16px 0">
 
 由Bilibili [土川虎虎虎](https://space.bilibili.com/29125536) 和 [明昭MZhao](https://space.bilibili.com/24807452)开发
 
@@ -9,58 +10,6 @@ description = """<div align="center" style="margin:16px 0">
 此App使用 `gpt-3.5-turbo` 大语言模型
 </div>
 """
-customCSS = """
-#status_display {
-    display: flex;
-    min-height: 2.5em;
-    align-items: flex-end;
-    justify-content: flex-end;
-}
-#status_display p {
-    font-size: .85em;
-    font-family: monospace;
-    color: var(--text-color-subdued) !important;
-}
-[class *= "message"] {
-    border-radius: var(--radius-xl) !important;
-    border: none;
-    padding: var(--spacing-xl) !important;
-    font-size: var(--text-md) !important;
-    line-height: var(--line-md) !important;
-}
-[data-testid = "bot"] {
-    max-width: 85%;
-    border-bottom-left-radius: 0 !important;
-}
-[data-testid = "user"] {
-    max-width: 85%;
-    width: auto !important;
-    border-bottom-right-radius: 0 !important;
-}
-code {
-    display: inline;
-    white-space: break-spaces;
-    border-radius: 6px;
-    margin: 0 2px 0 2px;
-    padding: .2em .4em .1em .4em;
-    background-color: rgba(175,184,193,0.2);
-}
-pre code {
-    display: block;
-    white-space: pre;
-    background-color: hsla(0, 0%, 0%, 72%);
-    border: solid 5px var(--color-border-primary) !important;
-    border-radius: 10px;
-    padding: 0 1.2rem 1.2rem;
-    margin-top: 1em !important;
-    color: #FFF;
-    box-shadow: inset 0px 8px 16px hsla(0, 0%, 0%, .2)
-}
-
-* {
-    transition: all 0.6s;
-}
-"""
 
 summarize_prompt = "你是谁？我们刚才聊了什么？"  # 总结对话时的 prompt
 MODELS = [
@@ -71,7 +20,9 @@ MODELS = [
     "gpt-4-32k",
     "gpt-4-32k-0314",
 ]  # 可选的模型
-websearch_prompt = """Web search results:
+
+websearch_prompt = """\
+Web search results:
 
 {web_results}
 Current date: {current_date}

requirements.txt

@@ -6,3 +6,4 @@ socksio
 tqdm
 colorama
 duckduckgo_search
+Pygments

utils.py

@@ -53,7 +53,7 @@ def postprocess(
             # None if message is None else markdown.markdown(message),
             # None if response is None else markdown.markdown(response),
             None if message is None else message,
-            None if response is None else mdtex2html.convert(response),
+            None if response is None else mdtex2html.convert(response, extensions=['fenced_code','codehilite','tables']),
         )
     return y
 
@@ -66,34 +66,19 @@ def count_token(message):
 
 
 def parse_text(text):
-    lines = text.split("\n")
-    lines = [line for line in lines if line != ""]
-    count = 0
-    for i, line in enumerate(lines):
-        if "```" in line:
-            count += 1
-            items = line.split("`")
-            if count % 2 == 1:
-                lines[i] = f'<pre><code class="language-{items[-1]}">'
-            else:
-                lines[i] = f"<br></code></pre>"
+    in_code_block = False
+    new_lines = []
+    for i,line in enumerate(text.split("\n")):
+        if line.strip().startswith("```"):
+            in_code_block = not in_code_block
+        if in_code_block:
+            if line.strip() != "":
+                new_lines.append(line)
         else:
-            if i > 0:
-                if count % 2 == 1:
-                    line = line.replace("`", "\`")
-                    line = line.replace("<", "&lt;")
-                    line = line.replace(">", "&gt;")
-                    line = line.replace(" ", "&nbsp;")
-                    line = line.replace("*", "&ast;")
-                    line = line.replace("_", "&lowbar;")
-                    line = line.replace("-", "&#45;")
-                    line = line.replace(".", "&#46;")
-                    line = line.replace("!", "&#33;")
-                    line = line.replace("(", "&#40;")
-                    line = line.replace(")", "&#41;")
-                    line = line.replace("$", "&#36;")
-                lines[i] = "<br>" + line
-    text = "".join(lines)
+            new_lines.append(line)
+    if in_code_block:
+        new_lines.append("```")
+    text = "\n".join(new_lines)
     return text