JunxiongWang/MambaByte_Books

Train in 30B Byte. Mode size 353M. Table 2 in MambaByte

To use

import torch
from mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel

import numpy as np

model=MambaLMHeadModel.from_pretrained("JunxiongWang/MambaByte_Books", device='cuda', dtype=torch.float32)

text = "In fair Verona, where we lay our scene, From ancient grudge, break to new mutiny, Where civil blood makes civil hands unclean."
text_byte = np.frombuffer(text.encode('utf-8'), dtype=np.uint8)
input_ids = torch.from_numpy(text_byte[None, :].copy()).long().cuda()

sample = model.generate(
    input_ids=input_ids,
    max_length=2048,
    cg=True,
    return_dict_in_generate=True,
    output_scores=True,
    enable_timing=True,
    temperature=1,
    top_k=256,
    top_p=0.9,
)

print(bytes(sample.sequences[0].tolist()).decode('utf-8'))

Output

In fair Verona, where we lay our scene, From ancient grudge, break to new mutiny, Where civil blood makes civil hands unclean. You are speaking truly, Marcheno . . . Here is a goodly show of arms we might have at you, That which is good we might do in each other's place. Your arms are too swift, O King, to make wars mean, too slow, Too fast without measure. What weapon could I wish that you and your brother should be better than this great king? I will buy such great arms as this, for this city where we lay our hands on them, made them our ends, Our hands . . ."

On the rack, among the shards of crockery, we found a part of the face of the old king. We spit and cut ourselves with these and fought the like battle, and no difference of style or color marked our heads, so that the gods should have fallen to their knees in prayer.

When we had gone away into the night, on our way back from the mountains of Montiagra, to dinner at the Palace of Ancestry, and back again to his chambers, Midas begged me to be so kind as to accept one of his books. I have bought it from the very best that the Princess of Orchis and I possess, and I am wondering that she should lend it to me. With her gifts are impressed the divinities of her forefathers, and with her own heart.

THIS evening, when we came into the room together, there stood the King surrounded by his wives, his mistresses, and his concubines, and all his people, all his women, and all his friends. Some of them dropped their heads at his feet, their eyes were blank, and their nostrils were red; some held their hands in awkward loops, and held their faces high.
As if they were sorry that they had not their true natures, they said to one another, "May the gods grant us this that we should not have been fortunate and sent these drowned children to our king to suffer." Some also said, "May the bride be with the groom too, for she hath the most wonderful thing in the world." But none of them was very anxious, because they did not know what they were to do. The King took his sw