topic_modelling / funcs /anonymiser.py

Commit History

Added example of how to run function from command line. Updated packages. Embedding model default now smaller and at fp16.
34f1e83

seanpedrickcase commited on

App now retains original index following cleaning to allow for referring back to original data
90553eb

seanpedrickcase commited on

Only aggregate topics not 'other', allowed for minimum sentence length, default max_topics now will auto aggregate topics. Added Cognito Auth functionality (boto3 with AWS).
1e2bb3e

seanpedrickcase commited on

Can split passages into sentences. Improved embedding, LLM representation models, improved zero shot capabilities
55f0ce3

seanpedrickcase commited on

Updated packages. Improve hierarchy vis. Better models - mixedbread and phi3. Now option to split texts into sentences before modelling.
04a15c5

seanpedrickcase commited on

Added clean data options, improved re-representation options and visualisation. General format changes
4effac0

Sonnyjim commited on

Lots of general fixes. New visualisations, fixed hierarchical vis for zero shot. Added calc all probabilities.
b4510a6

Sonnyjim commited on

Added option to reduce outliers based on closest topic
e09dd3b

Sonnyjim commited on

first commit
9dbf344

Sonnyjim commited on