llama3-qwantz-punchline

This model is a fine-tuned version of unsloth/llama-3-8b-bnb-4bit on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8012
• Accuracy: 0.7706

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.6763	1.0	300	0.7377	0.7529
0.419	2.0	600	0.8012	0.7706

Can save 50% of punchlines by discarding 97% of nonsequiturs (cutoff is 99.31640625)
Can save 75% of punchlines by discarding 95% of nonsequiturs (cutoff is 91.6015625)
Can save 90% of punchlines by discarding 77% of nonsequiturs (cutoff is -69.43359375)
Can save 92% of punchlines by discarding 62% of nonsequiturs (cutoff is -85.400390625)
Can save 93% of punchlines by discarding 61% of nonsequiturs (cutoff is -87.109375)
Can save 94% of punchlines by discarding 55% of nonsequiturs (cutoff is -92.28515625)
Can save 95% of punchlines by discarding 52% of nonsequiturs (cutoff is -94.482421875)
Can save 98% of punchlines by discarding 8% of nonsequiturs (cutoff is -99.70703125)
Can save 99% of punchlines by discarding 5% of nonsequiturs (cutoff is -99.8046875)

yor that says I'm not allowed to name anything ever! [SPEAKER] T-REX [LINE] Hah! Shakespeare's already QUITE the character, believe you me!!  ==>  nonsequitur: 88.18%
the mayor that says I'm not allowed to name anything ever! [SPEAKER] T-REX [LINE] THAT should finally straighten up some things around here!  ==>  real: 95.17%
SPEAKER] T-REX [LINE] You're welcome! [SPEAKER] T-REX [LINE] Because Utahraptor, it means that today is the first time in history, in ALL OF  ==>  nonsequitur: 96.92%
rk. [SPEAKER] T-REX [LINE] You're welcome! [SPEAKER] T-REX [LINE] Now, HERE'S how putting the things in my mouth ties in with existence. . .  ==>  real: 80.86%
reasing in a closed system is so WRONG. So... NAUGHTY. [SPEAKER] T-REX [LINE] Homecare is when doctors or nurses visit you in your own home!  ==>  nonsequitur: 99.90%
py decreasing in a closed system is so WRONG. So... NAUGHTY. [SPEAKER] T-REX [LINE] Aw, man!! Am I a dude who gets turned on by descrambling  ==>  nonsequitur: 55.22%
T-REX [LINE] Oh! Yeah, it was nice. We had "chai" tea. [SPEAKER] UTAHRAPTOR [LINE] And...? [SPEAKER] T-REX [LINE] It's a joke that plays off  ==>  nonsequitur: 99.32%
EX [LINE] Oh! Yeah, it was nice. We had "chai" tea. [SPEAKER] UTAHRAPTOR [LINE] And...? [SPEAKER] T-REX [LINE] And then we had dinosaur sex!  ==>  real: 99.61%
ent! [SPEAKER] FUTURE T-REX [LINE] Utahraptor! I'm gonna kiss you so much in ten minutes! [SPEAKER] T-REX [LINE] I was gonna do that anyway!  ==>  real: 99.95%
oment! [SPEAKER] FUTURE T-REX [LINE] Utahraptor! I'm gonna kiss you so much in ten minutes! [SPEAKER] T-REX [LINE] Um - that, ah, wasn't me.  ==>  real: 100.00%
end in victory... or in fiery death for us both? FIND OUT IN MY NEXT BOOK! [SPEAKER] T-REX [LINE] It's time to shake things up, T-Rex style!  ==>  nonsequitur: 96.97%
t end in victory... or in fiery death for us both? FIND OUT IN MY NEXT BOOK! [SPEAKER] T-REX [LINE] The book is called, "A Plane Saved Them,  ==>  nonsequitur: 85.40%
] NARRATOR [LINE] LATER, A LONG TRAIN RIDE HOME: [SPEAKER] T-REX [LINE] I also vacuum as if nobody's going to get their carpets any cleaner!  ==>  nonsequitur: 80.57%
 [SPEAKER] NARRATOR [LINE] LATER, A LONG TRAIN RIDE HOME: [SPEAKER] T-REX [LINE] "If there was a singular Dickens who had to pee, I would be  ==>  nonsequitur: 97.95%
with a candle and sang "Happy birthday" to myself, quiet and soft. [SPEAKER] T-REX [LINE] The only downside is that the Dromiceiomimus won't  ==>  nonsequitur: 78.32%
ce of cake with a candle and sang "Happy birthday" to myself, quiet and soft. [SPEAKER] T-REX [LINE] The waitress gave me the cake for free!  ==>  real: 99.90%
X [LINE] He probably would!  Man! [SPEAKER] T-REX [LINE] Poor Al! [SPEAKER] T-REX [LINE] Dromiceiomimus, do you ever approach something new,  ==>  nonsequitur: 98.83%
R] T-REX [LINE] He probably would!  Man! [SPEAKER] T-REX [LINE] Poor Al! [SPEAKER] T-REX [LINE] I've never felt worse for inventing someone!  ==>  real: 99.71%
on a helicopter ride. A ghost appears, and when Hamlet high fives him he explodes! Then Denmark explodes. [SPEAKER] T-REX [LINE] I AM MERELY  ==>  nonsequitur: 97.46%
r ride. A ghost appears, and when Hamlet high fives him he explodes! Then Denmark explodes. [SPEAKER] T-REX [LINE] Later, Ophelia's topless!  ==>  real: 99.90%
 my problems look like vegetables in need of a good steaming! [SPEAKER] T-REX [LINE] I've - [SPEAKER] T-REX [LINE] True stories! FACTS! Want  ==>  nonsequitur: 98.68%
 my problems look like vegetables in need of a good steaming! [SPEAKER] T-REX [LINE] I've - [SPEAKER] T-REX [LINE] I've made worse purchases  ==>  real: 98.19%
is' causes is your BOWELS becoming PERFORATED so that RAW SEWAGE sloshes around in your belly? [SPEAKER] T-REX [LINE] It's new and exciting!  ==>  real: 99.51%
is' causes is your BOWELS becoming PERFORATED so that RAW SEWAGE sloshes around in your belly? [SPEAKER] T-REX [LINE] Man, bodies are nasty!  ==>  real: 99.37%
injured, but they injure who they take care of." [SPEAKER] T-REX [LINE] Yes, really? Someone's name could be FARTNOISE FARTNOISE JUNIOR, and  ==>  nonsequitur: 99.80%
f the injured, but they injure who they take care of." [SPEAKER] T-REX [LINE] Okay, whatever! My formula is still good DESPITE nurse racism.  ==>  real: 99.95%
pin will ENTIRELY OVERSHADOW anything I've ever accomplished in my life! [SPEAKER] T-REX [LINE] It TOTALLY works! I just thought that losing  ==>  nonsequitur: 99.90%
the pin will ENTIRELY OVERSHADOW anything I've ever accomplished in my life! [SPEAKER] T-REX [LINE] No, that - that only makes things worse.  ==>  real: 99.85%
 TOE, turns to the camera in terror, wide-eyed, and screams "Oh GOSSSSSH!!!" [SPEAKER] T-REX [LINE] I got one that was like "I am really sad  ==>  nonsequitur: 99.56%
 to the camera in terror, wide-eyed, and screams "Oh GOSSSSSH!!!" [SPEAKER] T-REX [LINE] Holy crap. There's nothing not to like in my movie.  ==>  real: 99.85%
BILITY! IF WE WERE ALL FRIENDS, I AM CERTAIN YOU WOULD BE "THE IRRITATING ONE"! [SPEAKER] T-REX [LINE] Maybe you'd all like to come and have  ==>  nonsequitur: 77.25%
IF WE WERE ALL FRIENDS, I AM CERTAIN YOU WOULD BE "THE IRRITATING ONE"! [SPEAKER] T-REX [LINE] Oh my gosh, Utahraptor! I concede your point!  ==>  real: 100.00%
  i'm trying to write some sonnets here, t-rex! [SPEAKER] T-REX [LINE] HEY SHAKESPEARE! [SPEAKER] T-REX [LINE] It's not that hard, you guys!  ==>  nonsequitur: 99.02%
 trying to write some sonnets here, t-rex! [SPEAKER] T-REX [LINE] HEY SHAKESPEARE! [SPEAKER] T-REX [LINE] I DON'T THINK PEOPLE ACTUALLY USED  ==>  nonsequitur: 98.78%
rrelevant." He then focused himself on the seriously erotic business of being Hamlet II. [SPEAKER] T-REX [LINE] Debatable! People often talk  ==>  real: 75.59%
ow irrelevant." He then focused himself on the seriously erotic business of being Hamlet II. [SPEAKER] T-REX [LINE] There! Sex AND politics!  ==>  real: 94.38%
 Secret Mission of Wikipedia! Don't tell anyone, okay? [SPEAKER] T-REX [LINE] Hah! Awesome! [SPEAKER] T-REX [LINE] Imagine if we had a beach  ==>  nonsequitur: 99.85%
t Mission of Wikipedia! Don't tell anyone, okay? [SPEAKER] T-REX [LINE] Hah! Awesome! [SPEAKER] T-REX [LINE] ...Why didn't they just say so?  ==>  real: 99.95%
ns but his teeth passed right through brains!! [SPEAKER] UTAHRAPTOR [LINE] Oh yeah? [SPEAKER] T-REX [LINE] I'm tired of movies and plays and  ==>  nonsequitur: 99.32%
 teeth passed right through brains!! [SPEAKER] UTAHRAPTOR [LINE] Oh yeah? [SPEAKER] T-REX [LINE] I would describe him as "pretty frustrated"  ==>  real: 100.00%
I am loveable and trustworthy and good at conversations! [SPEAKER] T-REX [LINE] ... [SPEAKER] T-REX [LINE] Historically speaking, all I know  ==>  nonsequitur: 69.97%
loveable and trustworthy and good at conversations! [SPEAKER] T-REX [LINE] ... [SPEAKER] T-REX [LINE] I've had FIVE girlfriends, Utahraptor.  ==>  real: 99.90%

Framework versions

• PEFT 0.10.0
• Transformers 4.40.2
• Pytorch 2.2.1+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1