> Training Environment: | > Backend: Torch | > Mixed precision: False | > Precision: float32 | > Current device: 0 | > Num. of GPUs: 1 | > Num. of CPUs: 4 | > Num. of Torch Threads: 1 | > Torch seed: 54321 | > Torch CUDNN: True | > Torch CUDNN deterministic: False | > Torch CUDNN benchmark: False | > Torch TF32 MatMul: False > Start Tensorboard: tensorboard --logdir=/tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb > Model has 525252923 parameters [4m[1m > EPOCH: 0/8[0m --> /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb [1m > TRAINING (2024-12-19 13:24:34) [0m [1m --> TIME: 2024-12-19 13:24:37 -- STEP: 0/4076 -- GLOBAL_STEP: 0[0m | > loss_text_ce: 0.05042382702231407 (0.05042382702231407) | > loss_mel_ce: 4.394047260284424 (4.394047260284424) | > loss: 1.1111177206039429 (1.1111177206039429) | > current_lr: 5e-06 | > step_time: 0.87 (0.8700447082519531) | > loader_time: 2.4254 (2.4254345893859863) [1m --> TIME: 2024-12-19 13:24:58 -- STEP: 50/4076 -- GLOBAL_STEP: 50[0m | > loss_text_ce: 0.05801791697740555 (0.051985438466072086) | > loss_mel_ce: 4.11599588394165 (4.254919385910032) | > loss: 1.0435034036636353 (1.076726212501526) | > current_lr: 5e-06 | > step_time: 0.2838 (0.2299114227294922) | > loader_time: 0.0129 (0.01363321304321289) [1m --> TIME: 2024-12-19 13:25:19 -- STEP: 100/4076 -- GLOBAL_STEP: 100[0m | > loss_text_ce: 0.04592360928654671 (0.05045760903507471) | > loss_mel_ce: 4.447800636291504 (4.115956647396087) | > loss: 1.1234310865402222 (1.041603568196297) | > current_lr: 5e-06 | > step_time: 0.2323 (0.2305515241622925) | > loader_time: 0.0085 (0.012788057327270508) [1m --> TIME: 2024-12-19 13:25:40 -- STEP: 150/4076 -- GLOBAL_STEP: 150[0m | > loss_text_ce: 0.043412946164608 (0.04942907186845938) | > loss_mel_ce: 3.8673102855682373 (4.023904592196146) | > loss: 0.9776808023452759 (1.0183334183692925) | > current_lr: 5e-06 | > step_time: 0.2089 (0.22928651968638103) | > loader_time: 0.0127 (0.012653945287068683) [1m --> TIME: 2024-12-19 13:26:01 -- STEP: 200/4076 -- GLOBAL_STEP: 200[0m | > loss_text_ce: 0.037905123084783554 (0.04861956179141998) | > loss_mel_ce: 4.156767845153809 (3.966739008426666) | > loss: 1.0486682653427124 (1.0038396459817882) | > current_lr: 5e-06 | > step_time: 0.1582 (0.22992080450057983) | > loader_time: 0.0079 (0.012244981527328492) [1m --> TIME: 2024-12-19 13:26:22 -- STEP: 250/4076 -- GLOBAL_STEP: 250[0m | > loss_text_ce: 0.040389373898506165 (0.04792359894514083) | > loss_mel_ce: 3.8946244716644287 (3.921487903594971) | > loss: 0.9837534427642822 (0.9923528783321376) | > current_lr: 5e-06 | > step_time: 0.2867 (0.22969749355316163) | > loader_time: 0.0115 (0.01184248447418213) [1m --> TIME: 2024-12-19 13:26:43 -- STEP: 300/4076 -- GLOBAL_STEP: 300[0m | > loss_text_ce: 0.0471767894923687 (0.04741797216236591) | > loss_mel_ce: 3.3419079780578613 (3.8807986267407735) | > loss: 0.847271203994751 (0.982054152290026) | > current_lr: 5e-06 | > step_time: 0.1755 (0.228866978486379) | > loader_time: 0.0095 (0.011463300387064614) [1m --> TIME: 2024-12-19 13:27:04 -- STEP: 350/4076 -- GLOBAL_STEP: 350[0m | > loss_text_ce: 0.04379257932305336 (0.047042652634637715) | > loss_mel_ce: 3.719482898712158 (3.844630351747785) | > loss: 0.9408188462257385 (0.9729182532855439) | > current_lr: 5e-06 | > step_time: 0.2284 (0.2292230497087751) | > loader_time: 0.0085 (0.01123602049691337) [1m --> TIME: 2024-12-19 13:27:25 -- STEP: 400/4076 -- GLOBAL_STEP: 400[0m | > loss_text_ce: 0.04128987714648247 (0.046594090191647425) | > loss_mel_ce: 3.8073627948760986 (3.8206794667243957) | > loss: 0.9621631503105164 (0.9668183910846707) | > current_lr: 5e-06 | > step_time: 0.2085 (0.22927183628082276) | > loader_time: 0.008 (0.011069855690002446) [1m --> TIME: 2024-12-19 13:27:46 -- STEP: 450/4076 -- GLOBAL_STEP: 450[0m | > loss_text_ce: 0.04007521644234657 (0.04628008021248713) | > loss_mel_ce: 3.4973762035369873 (3.8033106390635174) | > loss: 0.8843628764152527 (0.9623976816071402) | > current_lr: 5e-06 | > step_time: 0.3581 (0.22916454739040798) | > loader_time: 0.0106 (0.010845611360338002) [1m --> TIME: 2024-12-19 13:28:06 -- STEP: 500/4076 -- GLOBAL_STEP: 500[0m | > loss_text_ce: 0.04251125827431679 (0.04602536760270597) | > loss_mel_ce: 3.485912561416626 (3.785600270748138) | > loss: 0.8821059465408325 (0.9579064111709592) | > current_lr: 5e-06 | > step_time: 0.2159 (0.22847333145141602) | > loader_time: 0.0082 (0.010652364253997806) [1m --> TIME: 2024-12-19 13:28:27 -- STEP: 550/4076 -- GLOBAL_STEP: 550[0m | > loss_text_ce: 0.04305431246757507 (0.045780698318373086) | > loss_mel_ce: 3.5659382343292236 (3.764264387650923) | > loss: 0.9022481441497803 (0.9525112729722803) | > current_lr: 5e-06 | > step_time: 0.2504 (0.2279539949243719) | > loader_time: 0.0097 (0.01052374276247892) [1m --> TIME: 2024-12-19 13:28:47 -- STEP: 600/4076 -- GLOBAL_STEP: 600[0m | > loss_text_ce: 0.046033646911382675 (0.045583339836448446) | > loss_mel_ce: 3.6808719635009766 (3.7507884657382964) | > loss: 0.9317263960838318 (0.94909295241038) | > current_lr: 5e-06 | > step_time: 0.1924 (0.22755316257476801) | > loader_time: 0.0083 (0.01040940920511882) [1m --> TIME: 2024-12-19 13:29:09 -- STEP: 650/4076 -- GLOBAL_STEP: 650[0m | > loss_text_ce: 0.042678456753492355 (0.04534391705806438) | > loss_mel_ce: 3.6471850872039795 (3.7404018515806934) | > loss: 0.9224658608436584 (0.9464364425952618) | > current_lr: 5e-06 | > step_time: 0.2599 (0.22823490179502043) | > loader_time: 0.0082 (0.010320220360389128) [1m --> TIME: 2024-12-19 13:29:30 -- STEP: 700/4076 -- GLOBAL_STEP: 700[0m | > loss_text_ce: 0.033985886722803116 (0.04521563749760389) | > loss_mel_ce: 3.52146315574646 (3.72840464421681) | > loss: 0.8888622522354126 (0.9434050704751696) | > current_lr: 5e-06 | > step_time: 0.1566 (0.22829034703118456) | > loader_time: 0.01 (0.010233897141047887) [1m --> TIME: 2024-12-19 13:29:51 -- STEP: 750/4076 -- GLOBAL_STEP: 750[0m | > loss_text_ce: 0.04124816507101059 (0.045020213117202126) | > loss_mel_ce: 3.517673969268799 (3.715415170987447) | > loss: 0.8897305130958557 (0.9401088458697001) | > current_lr: 5e-06 | > step_time: 0.2853 (0.2283107970555623) | > loader_time: 0.0081 (0.010196549097696935) [1m --> TIME: 2024-12-19 13:30:12 -- STEP: 800/4076 -- GLOBAL_STEP: 800[0m | > loss_text_ce: 0.04339027404785156 (0.044852362079545856) | > loss_mel_ce: 3.2865428924560547 (3.7057103994488716) | > loss: 0.8324832916259766 (0.9376406904309988) | > current_lr: 5e-06 | > step_time: 0.1553 (0.22876900553703305) | > loader_time: 0.0081 (0.010168586671352377) [1m --> TIME: 2024-12-19 13:30:33 -- STEP: 850/4076 -- GLOBAL_STEP: 850[0m | > loss_text_ce: 0.04460502415895462 (0.04467669516801834) | > loss_mel_ce: 3.5844826698303223 (3.69593169520883) | > loss: 0.9072719216346741 (0.9351520975898294) | > current_lr: 5e-06 | > step_time: 0.2618 (0.22904645330765666) | > loader_time: 0.0092 (0.010119463415706851) [1m --> TIME: 2024-12-19 13:30:54 -- STEP: 900/4076 -- GLOBAL_STEP: 900[0m | > loss_text_ce: 0.04466862604022026 (0.04453302537815439) | > loss_mel_ce: 3.5649843215942383 (3.6888919909795126) | > loss: 0.9024132490158081 (0.9333562541670269) | > current_lr: 5e-06 | > step_time: 0.1577 (0.2285395765304565) | > loader_time: 0.0081 (0.010040050877465134) [1m --> TIME: 2024-12-19 13:31:14 -- STEP: 950/4076 -- GLOBAL_STEP: 950[0m | > loss_text_ce: 0.0428137332201004 (0.04437729115941023) | > loss_mel_ce: 3.4124937057495117 (3.678411885311729) | > loss: 0.8638268709182739 (0.930697294473648) | > current_lr: 5e-06 | > step_time: 0.23 (0.2284435166810688) | > loader_time: 0.0097 (0.009998605376795712) [1m --> TIME: 2024-12-19 13:31:36 -- STEP: 1000/4076 -- GLOBAL_STEP: 1000[0m | > loss_text_ce: 0.04281843453645706 (0.0442620296292007) | > loss_mel_ce: 3.207716941833496 (3.67048095870018) | > loss: 0.8126338720321655 (0.9286857475042343) | > current_lr: 5e-06 | > step_time: 0.1444 (0.22871685719490048) | > loader_time: 0.0116 (0.009973724842071527) [1m --> TIME: 2024-12-19 13:31:57 -- STEP: 1050/4076 -- GLOBAL_STEP: 1050[0m | > loss_text_ce: 0.04180129989981651 (0.04413465928818498) | > loss_mel_ce: 3.2817721366882324 (3.6625097020467123) | > loss: 0.830893337726593 (0.92666109085083) | > current_lr: 5e-06 | > step_time: 0.2128 (0.22875983942122685) | > loader_time: 0.0096 (0.009942419188363203) [1m --> TIME: 2024-12-19 13:32:17 -- STEP: 1100/4076 -- GLOBAL_STEP: 1100[0m | > loss_text_ce: 0.04344223439693451 (0.044011552462523626) | > loss_mel_ce: 3.8075873851776123 (3.6552728704972703) | > loss: 0.962757408618927 (0.924821106141264) | > current_lr: 5e-06 | > step_time: 0.2106 (0.22851155172694812) | > loader_time: 0.0081 (0.009922269040887999) [1m --> TIME: 2024-12-19 13:32:38 -- STEP: 1150/4076 -- GLOBAL_STEP: 1150[0m | > loss_text_ce: 0.0372680127620697 (0.043929092968287675) | > loss_mel_ce: 3.422194480895996 (3.648275286840356) | > loss: 0.8648656010627747 (0.9230510953198312) | > current_lr: 5e-06 | > step_time: 0.2498 (0.22825955847035284) | > loader_time: 0.0082 (0.009886233288308843) [1m --> TIME: 2024-12-19 13:32:59 -- STEP: 1200/4076 -- GLOBAL_STEP: 1200[0m | > loss_text_ce: 0.042510610073804855 (0.04379834291214744) | > loss_mel_ce: 3.2973499298095703 (3.642642779747645) | > loss: 0.8349651098251343 (0.9216102807720506) | > current_lr: 5e-06 | > step_time: 0.1791 (0.2285618168115616) | > loader_time: 0.0093 (0.009878574212392167) [1m --> TIME: 2024-12-19 13:33:20 -- STEP: 1250/4076 -- GLOBAL_STEP: 1250[0m | > loss_text_ce: 0.041593652218580246 (0.04371409288048744) | > loss_mel_ce: 3.586989164352417 (3.638195256614685) | > loss: 0.9071456789970398 (0.9204773375511173) | > current_lr: 5e-06 | > step_time: 0.3193 (0.22864338932037354) | > loader_time: 0.0101 (0.009839678192138678) [1m --> TIME: 2024-12-19 13:33:42 -- STEP: 1300/4076 -- GLOBAL_STEP: 1300[0m | > loss_text_ce: 0.03798745945096016 (0.04360397936059878) | > loss_mel_ce: 3.7538180351257324 (3.63331933259964) | > loss: 0.9479513764381409 (0.9192308281018188) | > current_lr: 5e-06 | > step_time: 0.1762 (0.2289889585054838) | > loader_time: 0.0082 (0.009814558946169331) [1m --> TIME: 2024-12-19 13:34:03 -- STEP: 1350/4076 -- GLOBAL_STEP: 1350[0m | > loss_text_ce: 0.041432078927755356 (0.04350955657660959) | > loss_mel_ce: 3.3812811374664307 (3.626912402047051) | > loss: 0.8556783199310303 (0.9176054898014778) | > current_lr: 5e-06 | > step_time: 0.2068 (0.22900818489215993) | > loader_time: 0.0097 (0.009786473026982054) [1m --> TIME: 2024-12-19 13:34:24 -- STEP: 1400/4076 -- GLOBAL_STEP: 1400[0m | > loss_text_ce: 0.0394282341003418 (0.04343387865594454) | > loss_mel_ce: 3.7509264945983887 (3.6208501490524836) | > loss: 0.9475886821746826 (0.9160710070814408) | > current_lr: 5e-06 | > step_time: 0.1557 (0.22903064387185232) | > loader_time: 0.0097 (0.00977486865861074) [1m --> TIME: 2024-12-19 13:34:44 -- STEP: 1450/4076 -- GLOBAL_STEP: 1450[0m | > loss_text_ce: 0.045633506029844284 (0.04337226256214334) | > loss_mel_ce: 3.394176959991455 (3.616804031668038) | > loss: 0.8599526286125183 (0.9150440738118931) | > current_lr: 5e-06 | > step_time: 0.211 (0.22884441359289762) | > loader_time: 0.0084 (0.00974911196478481) [1m --> TIME: 2024-12-19 13:35:05 -- STEP: 1500/4076 -- GLOBAL_STEP: 1500[0m | > loss_text_ce: 0.04330592602491379 (0.0432849341407418) | > loss_mel_ce: 3.560570001602173 (3.6109823252360025) | > loss: 0.9009689688682556 (0.9135668149789178) | > current_lr: 5e-06 | > step_time: 0.1742 (0.22899588425954184) | > loader_time: 0.0082 (0.009735839207967114) [1m --> TIME: 2024-12-19 13:35:26 -- STEP: 1550/4076 -- GLOBAL_STEP: 1550[0m | > loss_text_ce: 0.04659786447882652 (0.043230749121116026) | > loss_mel_ce: 3.0712435245513916 (3.6072754301563386) | > loss: 0.7794603705406189 (0.9126265449293202) | > current_lr: 5e-06 | > step_time: 0.1797 (0.2289007983669158) | > loader_time: 0.0092 (0.009722981145305017) [1m --> TIME: 2024-12-19 13:35:47 -- STEP: 1600/4076 -- GLOBAL_STEP: 1600[0m | > loss_text_ce: 0.04212119057774544 (0.0431545029650442) | > loss_mel_ce: 3.488905191421509 (3.602350297868252) | > loss: 0.8827565908432007 (0.9113762002065781) | > current_lr: 5e-06 | > step_time: 0.1918 (0.22885285973548888) | > loader_time: 0.0095 (0.009698199033737179) [1m --> TIME: 2024-12-19 13:36:08 -- STEP: 1650/4076 -- GLOBAL_STEP: 1650[0m | > loss_text_ce: 0.04422586411237717 (0.04308158487758851) | > loss_mel_ce: 3.405642509460449 (3.597461124766957) | > loss: 0.8624671101570129 (0.9101356774387941) | > current_lr: 5e-06 | > step_time: 0.2744 (0.22889344330989952) | > loader_time: 0.0089 (0.00967194759484494) [1m --> TIME: 2024-12-19 13:36:29 -- STEP: 1700/4076 -- GLOBAL_STEP: 1700[0m | > loss_text_ce: 0.03855780512094498 (0.04301642271963987) | > loss_mel_ce: 3.6058995723724365 (3.59439385582419) | > loss: 0.9111143350601196 (0.9093525697904478) | > current_lr: 5e-06 | > step_time: 0.1748 (0.22871511515449075) | > loader_time: 0.0095 (0.009647895728840556) [1m --> TIME: 2024-12-19 13:36:49 -- STEP: 1750/4076 -- GLOBAL_STEP: 1750[0m | > loss_text_ce: 0.05171879008412361 (0.042965393036603905) | > loss_mel_ce: 3.7824618816375732 (3.590605321747916) | > loss: 0.9585451483726501 (0.9083926788057602) | > current_lr: 5e-06 | > step_time: 0.1824 (0.22862295150756837) | > loader_time: 0.0077 (0.009630388259887704) [1m --> TIME: 2024-12-19 13:37:10 -- STEP: 1800/4076 -- GLOBAL_STEP: 1800[0m | > loss_text_ce: 0.038983054459095 (0.0429074157008694) | > loss_mel_ce: 3.2270700931549072 (3.586554309527079) | > loss: 0.8165132999420166 (0.9073654314544468) | > current_lr: 5e-06 | > step_time: 0.1733 (0.22857937243249682) | > loader_time: 0.0101 (0.00961218688223098) [1m --> TIME: 2024-12-19 13:37:30 -- STEP: 1850/4076 -- GLOBAL_STEP: 1850[0m | > loss_text_ce: 0.04102117568254471 (0.04286805197999282) | > loss_mel_ce: 3.464388132095337 (3.5834421532862897) | > loss: 0.8763523101806641 (0.90657755159043) | > current_lr: 5e-06 | > step_time: 0.2324 (0.2284518067901199) | > loader_time: 0.0081 (0.009596026137068473) [1m --> TIME: 2024-12-19 13:37:51 -- STEP: 1900/4076 -- GLOBAL_STEP: 1900[0m | > loss_text_ce: 0.04395190626382828 (0.04280183562518732) | > loss_mel_ce: 3.161350965499878 (3.579021564533836) | > loss: 0.8013257384300232 (0.905455850362778) | > current_lr: 5e-06 | > step_time: 0.1726 (0.22835830450057984) | > loader_time: 0.0082 (0.009577055228383926) [1m --> TIME: 2024-12-19 13:38:11 -- STEP: 1950/4076 -- GLOBAL_STEP: 1950[0m | > loss_text_ce: 0.04499328136444092 (0.042755413126104896) | > loss_mel_ce: 3.5428740978240967 (3.575467466574449) | > loss: 0.896966814994812 (0.9045557202436988) | > current_lr: 5e-06 | > step_time: 0.18 (0.22813821034553725) | > loader_time: 0.0078 (0.009553240996140707) [1m --> TIME: 2024-12-19 13:38:32 -- STEP: 2000/4076 -- GLOBAL_STEP: 2000[0m | > loss_text_ce: 0.04101094231009483 (0.04270762600190935) | > loss_mel_ce: 3.3927836418151855 (3.572292323112488) | > loss: 0.8584486246109009 (0.9037499874532225) | > current_lr: 5e-06 | > step_time: 0.2333 (0.22800343728065492) | > loader_time: 0.0084 (0.009531232714653021) [1m --> TIME: 2024-12-19 13:38:52 -- STEP: 2050/4076 -- GLOBAL_STEP: 2050[0m | > loss_text_ce: 0.04179307445883751 (0.042662688319276) | > loss_mel_ce: 3.715028762817383 (3.570467446955239) | > loss: 0.9392054677009583 (0.9032825339421996) | > current_lr: 5e-06 | > step_time: 0.2275 (0.22789607873777065) | > loader_time: 0.0083 (0.00951838225853153) [1m --> TIME: 2024-12-19 13:39:13 -- STEP: 2100/4076 -- GLOBAL_STEP: 2100[0m | > loss_text_ce: 0.044639866799116135 (0.04261894938136849) | > loss_mel_ce: 3.4512686729431152 (3.566909554118202) | > loss: 0.8739771246910095 (0.9023821260247914) | > current_lr: 5e-06 | > step_time: 0.1732 (0.2277282786369324) | > loader_time: 0.0088 (0.00950217349188669) [1m --> TIME: 2024-12-19 13:39:34 -- STEP: 2150/4076 -- GLOBAL_STEP: 2150[0m | > loss_text_ce: 0.04163983091711998 (0.04257677360676056) | > loss_mel_ce: 3.5677525997161865 (3.5641311582299173) | > loss: 0.9023481011390686 (0.9016769831956823) | > current_lr: 5e-06 | > step_time: 0.2095 (0.22783956461174545) | > loader_time: 0.0105 (0.009484104223029565) [1m --> TIME: 2024-12-19 13:39:55 -- STEP: 2200/4076 -- GLOBAL_STEP: 2200[0m | > loss_text_ce: 0.03945162892341614 (0.04254083483395251) | > loss_mel_ce: 3.439647674560547 (3.5608249327269474) | > loss: 0.8697748184204102 (0.9008414422382016) | > current_lr: 5e-06 | > step_time: 0.2353 (0.22787048329006546) | > loader_time: 0.0093 (0.009478639472614642) [1m --> TIME: 2024-12-19 13:40:15 -- STEP: 2250/4076 -- GLOBAL_STEP: 2250[0m | > loss_text_ce: 0.04376872256398201 (0.04248956529464988) | > loss_mel_ce: 3.4092190265655518 (3.5580434206856624) | > loss: 0.8632469177246094 (0.9001332467397063) | > current_lr: 5e-06 | > step_time: 0.2275 (0.22783138042026096) | > loader_time: 0.0087 (0.009470365736219627) [1m --> TIME: 2024-12-19 13:40:36 -- STEP: 2300/4076 -- GLOBAL_STEP: 2300[0m | > loss_text_ce: 0.04274676367640495 (0.042449911084512015) | > loss_mel_ce: 3.6258652210235596 (3.5553035154549977) | > loss: 0.917153000831604 (0.8994383568867401) | > current_lr: 5e-06 | > step_time: 0.1926 (0.22779892382414443) | > loader_time: 0.0087 (0.009459259821021046) [1m --> TIME: 2024-12-19 13:40:57 -- STEP: 2350/4076 -- GLOBAL_STEP: 2350[0m | > loss_text_ce: 0.040248021483421326 (0.042415197382264964) | > loss_mel_ce: 3.6118481159210205 (3.55266180981981) | > loss: 0.9130240082740784 (0.8987692520466264) | > current_lr: 5e-06 | > step_time: 0.2277 (0.22777252887157684) | > loader_time: 0.0089 (0.009450477234860685) [1m --> TIME: 2024-12-19 13:41:18 -- STEP: 2400/4076 -- GLOBAL_STEP: 2400[0m | > loss_text_ce: 0.038468580693006516 (0.04235930960625412) | > loss_mel_ce: 3.1257987022399902 (3.549670127530893) | > loss: 0.7910668253898621 (0.8980073594301946) | > current_lr: 5e-06 | > step_time: 0.1588 (0.2277426635225614) | > loader_time: 0.0083 (0.009447752336661014) [1m --> TIME: 2024-12-19 13:41:39 -- STEP: 2450/4076 -- GLOBAL_STEP: 2450[0m | > loss_text_ce: 0.04104679077863693 (0.042319577547664564) | > loss_mel_ce: 3.162700653076172 (3.5467687526041147) | > loss: 0.8009368777275085 (0.8972720826888578) | > current_lr: 5e-06 | > step_time: 0.2286 (0.22778330043870576) | > loader_time: 0.0108 (0.009434579732466714) [1m --> TIME: 2024-12-19 13:42:00 -- STEP: 2500/4076 -- GLOBAL_STEP: 2500[0m | > loss_text_ce: 0.03650889918208122 (0.042277333812415586) | > loss_mel_ce: 3.2566561698913574 (3.5439543254852297) | > loss: 0.8232912421226501 (0.8965579149723061) | > current_lr: 5e-06 | > step_time: 0.1559 (0.22779787359237672) | > loader_time: 0.0083 (0.009434591197967529) [1m --> TIME: 2024-12-19 13:42:20 -- STEP: 2550/4076 -- GLOBAL_STEP: 2550[0m | > loss_text_ce: 0.04473469778895378 (0.04224542627147596) | > loss_mel_ce: 3.5641860961914062 (3.541400419029535) | > loss: 0.9022302031517029 (0.895911461488874) | > current_lr: 5e-06 | > step_time: 0.1846 (0.2277730521033792) | > loader_time: 0.0097 (0.009434458695205988) [1m --> TIME: 2024-12-19 13:42:41 -- STEP: 2600/4076 -- GLOBAL_STEP: 2600[0m | > loss_text_ce: 0.03925738483667374 (0.042203488729607615) | > loss_mel_ce: 3.9343676567077637 (3.5391312093001144) | > loss: 0.9934062361717224 (0.8953336746188317) | > current_lr: 5e-06 | > step_time: 0.1913 (0.22782344286258405) | > loader_time: 0.0083 (0.00943365619732784) [1m --> TIME: 2024-12-19 13:43:02 -- STEP: 2650/4076 -- GLOBAL_STEP: 2650[0m | > loss_text_ce: 0.03605753183364868 (0.04215570087983924) | > loss_mel_ce: 3.7743217945098877 (3.536595088850777) | > loss: 0.9525948166847229 (0.8946876975275443) | > current_lr: 5e-06 | > step_time: 0.2078 (0.22783852010403038) | > loader_time: 0.009 (0.009431529045104992) [1m --> TIME: 2024-12-19 13:43:24 -- STEP: 2700/4076 -- GLOBAL_STEP: 2700[0m | > loss_text_ce: 0.03828035295009613 (0.04210659902680805) | > loss_mel_ce: 3.956019639968872 (3.5335146707958645) | > loss: 0.99857497215271 (0.8939053175846743) | > current_lr: 5e-06 | > step_time: 0.2359 (0.22792079236772325) | > loader_time: 0.0108 (0.009426696035597072) [1m --> TIME: 2024-12-19 13:43:44 -- STEP: 2750/4076 -- GLOBAL_STEP: 2750[0m | > loss_text_ce: 0.040651965886354446 (0.04206812747771094) | > loss_mel_ce: 3.741621255874634 (3.5307950295535) | > loss: 0.945568323135376 (0.8932157893614342) | > current_lr: 5e-06 | > step_time: 0.2073 (0.22789144611358642) | > loader_time: 0.0085 (0.00942235452478584) [1m --> TIME: 2024-12-19 13:44:05 -- STEP: 2800/4076 -- GLOBAL_STEP: 2800[0m | > loss_text_ce: 0.03931540250778198 (0.042026620465996034) | > loss_mel_ce: 3.584219455718994 (3.5281090941599436) | > loss: 0.9058837294578552 (0.8925339287093714) | > current_lr: 5e-06 | > step_time: 0.2338 (0.2277780020236969) | > loader_time: 0.0093 (0.009418239423206892) [1m --> TIME: 2024-12-19 13:44:25 -- STEP: 2850/4076 -- GLOBAL_STEP: 2850[0m | > loss_text_ce: 0.03992871940135956 (0.04199210163532644) | > loss_mel_ce: 3.494532585144043 (3.5262321215345147) | > loss: 0.8836153149604797 (0.8920560558009573) | > current_lr: 5e-06 | > step_time: 0.2108 (0.22765694450913815) | > loader_time: 0.008 (0.009413634768703544) [1m --> TIME: 2024-12-19 13:44:47 -- STEP: 2900/4076 -- GLOBAL_STEP: 2900[0m | > loss_text_ce: 0.044757794588804245 (0.04196813554213998) | > loss_mel_ce: 3.2615482807159424 (3.5238876951151883) | > loss: 0.8265765309333801 (0.8914639577166794) | > current_lr: 5e-06 | > step_time: 0.1768 (0.22779711082063872) | > loader_time: 0.0087 (0.009410213437573672) [1m --> TIME: 2024-12-19 13:45:07 -- STEP: 2950/4076 -- GLOBAL_STEP: 2950[0m | > loss_text_ce: 0.03787756338715553 (0.04192607152512522) | > loss_mel_ce: 3.438279867172241 (3.5222733431347346) | > loss: 0.8690393567085266 (0.891049853805769) | > current_lr: 5e-06 | > step_time: 0.1869 (0.2278219042794179) | > loader_time: 0.0099 (0.009411549891455716) [1m --> TIME: 2024-12-19 13:45:28 -- STEP: 3000/4076 -- GLOBAL_STEP: 3000[0m | > loss_text_ce: 0.03981179744005203 (0.041887830343718374) | > loss_mel_ce: 4.022050857543945 (3.520879936536153) | > loss: 1.0154656171798706 (0.8906919419169432) | > current_lr: 5e-06 | > step_time: 0.2412 (0.2277582852045695) | > loader_time: 0.0088 (0.009408938090006525) [1m --> TIME: 2024-12-19 13:45:49 -- STEP: 3050/4076 -- GLOBAL_STEP: 3050[0m | > loss_text_ce: 0.039715420454740524 (0.04184962727679097) | > loss_mel_ce: 3.5178096294403076 (3.5184432423701053) | > loss: 0.8893812894821167 (0.8900732176030275) | > current_lr: 5e-06 | > step_time: 0.2075 (0.22771902772246813) | > loader_time: 0.0085 (0.00940470429717519) [1m --> TIME: 2024-12-19 13:46:10 -- STEP: 3100/4076 -- GLOBAL_STEP: 3100[0m | > loss_text_ce: 0.040225714445114136 (0.04181272101558508) | > loss_mel_ce: 3.2103350162506104 (3.516664171295781) | > loss: 0.8126401901245117 (0.8896192232062745) | > current_lr: 5e-06 | > step_time: 0.1757 (0.22770175926146968) | > loader_time: 0.0085 (0.00940054278219901) [1m --> TIME: 2024-12-19 13:46:31 -- STEP: 3150/4076 -- GLOBAL_STEP: 3150[0m | > loss_text_ce: 0.04344997927546501 (0.04178045005374961) | > loss_mel_ce: 3.6015677452087402 (3.5149308359812177) | > loss: 0.9112544059753418 (0.8891778216475538) | > current_lr: 5e-06 | > step_time: 0.2131 (0.22781546350509402) | > loader_time: 0.0098 (0.009403231106107211) [1m --> TIME: 2024-12-19 13:46:52 -- STEP: 3200/4076 -- GLOBAL_STEP: 3200[0m | > loss_text_ce: 0.04286840185523033 (0.041750318037229624) | > loss_mel_ce: 3.168696880340576 (3.5127250526845457) | > loss: 0.8028913140296936 (0.8886188427545136) | > current_lr: 5e-06 | > step_time: 0.156 (0.22779312871396543) | > loader_time: 0.0085 (0.00939768418669702) [1m --> TIME: 2024-12-19 13:47:13 -- STEP: 3250/4076 -- GLOBAL_STEP: 3250[0m | > loss_text_ce: 0.04334869608283043 (0.041726415239274534) | > loss_mel_ce: 3.419900894165039 (3.510880239560054) | > loss: 0.8658124208450317 (0.8881516637985528) | > current_lr: 5e-06 | > step_time: 0.2324 (0.2279514033977802) | > loader_time: 0.0084 (0.009399798393249535) [1m --> TIME: 2024-12-19 13:47:35 -- STEP: 3300/4076 -- GLOBAL_STEP: 3300[0m | > loss_text_ce: 0.036624401807785034 (0.04170307497352817) | > loss_mel_ce: 3.5632314682006836 (3.5096151399612427) | > loss: 0.8999639749526978 (0.8878295538642196) | > current_lr: 5e-06 | > step_time: 0.2711 (0.2280530895608844) | > loader_time: 0.0105 (0.009403154777758068) [1m --> TIME: 2024-12-19 13:47:56 -- STEP: 3350/4076 -- GLOBAL_STEP: 3350[0m | > loss_text_ce: 0.04278753325343132 (0.04167000335821913) | > loss_mel_ce: 3.532724142074585 (3.507870414363804) | > loss: 0.8938779234886169 (0.8873851045387899) | > current_lr: 5e-06 | > step_time: 0.2644 (0.22807652188770808) | > loader_time: 0.0092 (0.009401319133701628) [1m --> TIME: 2024-12-19 13:48:17 -- STEP: 3400/4076 -- GLOBAL_STEP: 3400[0m | > loss_text_ce: 0.04266428202390671 (0.041642332696629834) | > loss_mel_ce: 3.3252453804016113 (3.5060762668357177) | > loss: 0.8419774174690247 (0.8869296499736175) | > current_lr: 5e-06 | > step_time: 0.1929 (0.22809735585661495) | > loader_time: 0.0103 (0.009408766101388398) [1m --> TIME: 2024-12-19 13:48:38 -- STEP: 3450/4076 -- GLOBAL_STEP: 3450[0m | > loss_text_ce: 0.0475010871887207 (0.041614764911001176) | > loss_mel_ce: 3.665635108947754 (3.5039418434060137) | > loss: 0.9282840490341187 (0.886389152122581) | > current_lr: 5e-06 | > step_time: 0.2063 (0.22813169970028643) | > loader_time: 0.009 (0.00941015775652902) [1m --> TIME: 2024-12-19 13:48:59 -- STEP: 3500/4076 -- GLOBAL_STEP: 3500[0m | > loss_text_ce: 0.03821425139904022 (0.04158637329669938) | > loss_mel_ce: 3.4899230003356934 (3.5025215911865235) | > loss: 0.8820343017578125 (0.8860269911629818) | > current_lr: 5e-06 | > step_time: 0.2323 (0.2281410903249468) | > loader_time: 0.0086 (0.009406382560730009) [1m --> TIME: 2024-12-19 13:49:20 -- STEP: 3550/4076 -- GLOBAL_STEP: 3550[0m | > loss_text_ce: 0.03254983201622963 (0.041557160849407554) | > loss_mel_ce: 3.2776293754577637 (3.5004332218035845) | > loss: 0.8275448083877563 (0.8854975957098147) | > current_lr: 5e-06 | > step_time: 0.2816 (0.22817747780974482) | > loader_time: 0.0107 (0.009402892354508549) [1m --> TIME: 2024-12-19 13:49:41 -- STEP: 3600/4076 -- GLOBAL_STEP: 3600[0m | > loss_text_ce: 0.039810776710510254 (0.04153312314560433) | > loss_mel_ce: 3.187955856323242 (3.4989868872695498) | > loss: 0.8069416284561157 (0.885130002631082) | > current_lr: 5e-06 | > step_time: 0.2086 (0.22827554391490087) | > loader_time: 0.0104 (0.009397697183820964) [1m --> TIME: 2024-12-19 13:50:02 -- STEP: 3650/4076 -- GLOBAL_STEP: 3650[0m | > loss_text_ce: 0.03752746433019638 (0.04150128321645606) | > loss_mel_ce: 3.266296863555908 (3.496210316697212) | > loss: 0.8259561061859131 (0.8844278999714007) | > current_lr: 5e-06 | > step_time: 0.2286 (0.22827067022454248) | > loader_time: 0.0096 (0.009399222282514207) [1m --> TIME: 2024-12-19 13:50:24 -- STEP: 3700/4076 -- GLOBAL_STEP: 3700[0m | > loss_text_ce: 0.0406351275742054 (0.04146698444628638) | > loss_mel_ce: 3.1213037967681885 (3.495037887611905) | > loss: 0.7904847264289856 (0.8841262179935305) | > current_lr: 5e-06 | > step_time: 0.2102 (0.22842442576949662) | > loader_time: 0.009 (0.009395978966274804) [1m --> TIME: 2024-12-19 13:50:45 -- STEP: 3750/4076 -- GLOBAL_STEP: 3750[0m | > loss_text_ce: 0.03915800899267197 (0.04144157496442401) | > loss_mel_ce: 3.3197684288024902 (3.4932217926661173) | > loss: 0.8397316336631775 (0.8836658418973292) | > current_lr: 5e-06 | > step_time: 0.2055 (0.228526673634847) | > loader_time: 0.012 (0.009393801689147975) [1m --> TIME: 2024-12-19 13:51:06 -- STEP: 3800/4076 -- GLOBAL_STEP: 3800[0m | > loss_text_ce: 0.041055116802453995 (0.04142156417187502) | > loss_mel_ce: 3.6712124347686768 (3.4911872477280466) | > loss: 0.9280669093132019 (0.8831522029481441) | > current_lr: 5e-06 | > step_time: 0.2178 (0.2284578107532702) | > loader_time: 0.01 (0.009394234167902119) [1m --> TIME: 2024-12-19 13:51:27 -- STEP: 3850/4076 -- GLOBAL_STEP: 3850[0m | > loss_text_ce: 0.04169200733304024 (0.04139868716092469) | > loss_mel_ce: 3.2259371280670166 (3.489401571905458) | > loss: 0.8169072866439819 (0.8827000647551059) | > current_lr: 5e-06 | > step_time: 0.186 (0.22848923169173205) | > loader_time: 0.0097 (0.009398987448060694) [1m --> TIME: 2024-12-19 13:51:48 -- STEP: 3900/4076 -- GLOBAL_STEP: 3900[0m | > loss_text_ce: 0.03974117711186409 (0.04137904698220204) | > loss_mel_ce: 3.0021419525146484 (3.4875445005221244) | > loss: 0.7604708075523376 (0.8822308868628287) | > current_lr: 5e-06 | > step_time: 0.1925 (0.22843588358316666) | > loader_time: 0.0092 (0.009396689671736522) [1m --> TIME: 2024-12-19 13:52:09 -- STEP: 3950/4076 -- GLOBAL_STEP: 3950[0m | > loss_text_ce: 0.037178345024585724 (0.04135220268057497) | > loss_mel_ce: 3.1396853923797607 (3.4855965233452713) | > loss: 0.7942159175872803 (0.8817371815065799) | > current_lr: 5e-06 | > step_time: 0.2808 (0.22851355957079536) | > loader_time: 0.009 (0.00939636665054517) [1m --> TIME: 2024-12-19 13:52:30 -- STEP: 4000/4076 -- GLOBAL_STEP: 4000[0m | > loss_text_ce: 0.03765800595283508 (0.04132433380978185) | > loss_mel_ce: 3.305239677429199 (3.4843947005867957) | > loss: 0.835724413394928 (0.8814297586232428) | > current_lr: 5e-06 | > step_time: 0.1893 (0.2284715247154236) | > loader_time: 0.0101 (0.00939310902357104) [1m --> TIME: 2024-12-19 13:52:51 -- STEP: 4050/4076 -- GLOBAL_STEP: 4050[0m | > loss_text_ce: 0.04226308688521385 (0.04130290373837877) | > loss_mel_ce: 3.509979248046875 (3.482472480667962) | > loss: 0.8880605697631836 (0.8809438460991711) | > current_lr: 5e-06 | > step_time: 0.212 (0.22849182417363295) | > loader_time: 0.009 (0.009392406145731632) [1m > EVALUATION [0m [1m--> EVAL PERFORMANCE[0m | > avg_loader_time: 0.46961667904487026 [0m(+0) | > avg_loss_text_ce: 0.03813300998165058 [0m(+0) | > avg_loss_mel_ce: 3.3231053260656505 [0m(+0) | > avg_loss: 3.3612383420650778 [0m(+0) [4m[1m > EPOCH: 1/8[0m --> /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb [1m > TRAINING (2024-12-19 13:53:58) [0m [1m --> TIME: 2024-12-19 13:54:08 -- STEP: 24/4076 -- GLOBAL_STEP: 4100[0m | > loss_text_ce: 0.03769286349415779 (0.03911654542510708) | > loss_mel_ce: 3.2999966144561768 (3.3180872003237405) | > loss: 0.8344223499298096 (0.8393009354670843) | > current_lr: 5e-06 | > step_time: 0.1586 (0.21927388509114584) | > loader_time: 0.009 (0.009344836076100668) [1m --> TIME: 2024-12-19 13:54:29 -- STEP: 74/4076 -- GLOBAL_STEP: 4150[0m | > loss_text_ce: 0.03946463018655777 (0.0393138415104634) | > loss_mel_ce: 3.3845760822296143 (3.2756181407619165) | > loss: 0.8560101985931396 (0.8287329963735631) | > current_lr: 5e-06 | > step_time: 0.2493 (0.2238888353914828) | > loader_time: 0.01 (0.009460594203021075) [1m --> TIME: 2024-12-19 13:54:50 -- STEP: 124/4076 -- GLOBAL_STEP: 4200[0m | > loss_text_ce: 0.035260315984487534 (0.03931754779431128) | > loss_mel_ce: 3.339848756790161 (3.3262680595920933) | > loss: 0.8437772393226624 (0.8413964024474544) | > current_lr: 5e-06 | > step_time: 0.1944 (0.2265023935225702) | > loader_time: 0.0097 (0.0094462844633287) [1m --> TIME: 2024-12-19 13:55:11 -- STEP: 174/4076 -- GLOBAL_STEP: 4250[0m | > loss_text_ce: 0.04390213266015053 (0.03944368011735634) | > loss_mel_ce: 3.421980619430542 (3.324056634957763) | > loss: 0.8664706945419312 (0.8408750795769966) | > current_lr: 5e-06 | > step_time: 0.2774 (0.22787161120052996) | > loader_time: 0.009 (0.009425816864802922) [1m --> TIME: 2024-12-19 13:55:32 -- STEP: 224/4076 -- GLOBAL_STEP: 4300[0m | > loss_text_ce: 0.03656553104519844 (0.0393782302105267) | > loss_mel_ce: 3.1088006496429443 (3.335902433310236) | > loss: 0.7863415479660034 (0.8438201661088637) | > current_lr: 5e-06 | > step_time: 0.2104 (0.22864711497511184) | > loader_time: 0.0086 (0.009322415505136758) [1m --> TIME: 2024-12-19 13:55:53 -- STEP: 274/4076 -- GLOBAL_STEP: 4350[0m | > loss_text_ce: 0.03640015423297882 (0.03924694169231138) | > loss_mel_ce: 3.440911293029785 (3.33842636111879) | > loss: 0.8693278431892395 (0.8444183255634168) | > current_lr: 5e-06 | > step_time: 0.2089 (0.22919823218436136) | > loader_time: 0.0095 (0.009266097180164641) [1m --> TIME: 2024-12-19 13:56:14 -- STEP: 324/4076 -- GLOBAL_STEP: 4400[0m | > loss_text_ce: 0.04462020844221115 (0.039379405053217444) | > loss_mel_ce: 3.2565996646881104 (3.33862101148676) | > loss: 0.8253049850463867 (0.8445001038504235) | > current_lr: 5e-06 | > step_time: 0.1543 (0.2290904786851671) | > loader_time: 0.009 (0.009234387197612246) [1m --> TIME: 2024-12-19 13:56:35 -- STEP: 374/4076 -- GLOBAL_STEP: 4450[0m | > loss_text_ce: 0.03538015857338905 (0.03926036958109566) | > loss_mel_ce: 3.214585781097412 (3.3347511909862253) | > loss: 0.8124914765357971 (0.8435028898205986) | > current_lr: 5e-06 | > step_time: 0.1811 (0.2294114260750021) | > loader_time: 0.0107 (0.009250714179666287) [1m --> TIME: 2024-12-19 13:56:55 -- STEP: 424/4076 -- GLOBAL_STEP: 4500[0m | > loss_text_ce: 0.039015308022499084 (0.039227909549086705) | > loss_mel_ce: 3.2652604579925537 (3.336634115790421) | > loss: 0.8260689377784729 (0.8439655060756881) | > current_lr: 5e-06 | > step_time: 0.1767 (0.22785562389301803) | > loader_time: 0.0105 (0.009302888838750016) [1m --> TIME: 2024-12-19 13:57:16 -- STEP: 474/4076 -- GLOBAL_STEP: 4550[0m | > loss_text_ce: 0.03970075398683548 (0.03919996300767241) | > loss_mel_ce: 3.3408164978027344 (3.3348109390162213) | > loss: 0.8451293110847473 (0.8435027257299624) | > current_lr: 5e-06 | > step_time: 0.2318 (0.2279031890354076) | > loader_time: 0.0103 (0.009303542632090898) [1m --> TIME: 2024-12-19 13:57:37 -- STEP: 524/4076 -- GLOBAL_STEP: 4600[0m | > loss_text_ce: 0.0350114181637764 (0.039216398723594094) | > loss_mel_ce: 3.611823320388794 (3.33464110989607) | > loss: 0.9117087125778198 (0.8434643776134679) | > current_lr: 5e-06 | > step_time: 0.156 (0.22784801202875968) | > loader_time: 0.0086 (0.009314571173136475) [1m --> TIME: 2024-12-19 13:57:58 -- STEP: 574/4076 -- GLOBAL_STEP: 4650[0m | > loss_text_ce: 0.03592097386717796 (0.039254444498789454) | > loss_mel_ce: 3.229173421859741 (3.3323876674997672) | > loss: 0.81627357006073 (0.8429105285983468) | > current_lr: 5e-06 | > step_time: 0.2286 (0.22884749867774884) | > loader_time: 0.0105 (0.00928429153322758) [1m --> TIME: 2024-12-19 13:58:19 -- STEP: 624/4076 -- GLOBAL_STEP: 4700[0m | > loss_text_ce: 0.04241777956485748 (0.03930146183269339) | > loss_mel_ce: 3.37697434425354 (3.3332683356144477) | > loss: 0.8548480272293091 (0.843142450046845) | > current_lr: 5e-06 | > step_time: 0.1748 (0.22890001420791334) | > loader_time: 0.0103 (0.009283425334172372) [1m --> TIME: 2024-12-19 13:58:41 -- STEP: 674/4076 -- GLOBAL_STEP: 4750[0m | > loss_text_ce: 0.035883959382772446 (0.03931174545551622) | > loss_mel_ce: 3.4333581924438477 (3.3328028110792784) | > loss: 0.8673105239868164 (0.8430286396504864) | > current_lr: 5e-06 | > step_time: 0.2757 (0.22928749666015896) | > loader_time: 0.0104 (0.009303545032127679) [1m --> TIME: 2024-12-19 13:59:01 -- STEP: 724/4076 -- GLOBAL_STEP: 4800[0m | > loss_text_ce: 0.03925236314535141 (0.039320771000991225) | > loss_mel_ce: 3.2696146965026855 (3.3286633122691778) | > loss: 0.8272167444229126 (0.8419960212806312) | > current_lr: 5e-06 | > step_time: 0.1943 (0.22911774255952783) | > loader_time: 0.0098 (0.00931050764263006) [1m --> TIME: 2024-12-19 13:59:22 -- STEP: 774/4076 -- GLOBAL_STEP: 4850[0m | > loss_text_ce: 0.03843331336975098 (0.03931429423391819) | > loss_mel_ce: 2.9299066066741943 (3.326728606100845) | > loss: 0.7420849800109863 (0.8415107256528327) | > current_lr: 5e-06 | > step_time: 0.2103 (0.22912557488572075) | > loader_time: 0.0105 (0.009331260232654352) [1m --> TIME: 2024-12-19 13:59:43 -- STEP: 824/4076 -- GLOBAL_STEP: 4900[0m | > loss_text_ce: 0.04025153070688248 (0.03930056933765705) | > loss_mel_ce: 3.3275108337402344 (3.3254745695197454) | > loss: 0.8419405817985535 (0.8411937853665028) | > current_lr: 5e-06 | > step_time: 0.1573 (0.2291126178884969) | > loader_time: 0.0106 (0.009390686322184455) [1m --> TIME: 2024-12-19 14:00:05 -- STEP: 874/4076 -- GLOBAL_STEP: 4950[0m | > loss_text_ce: 0.03222297877073288 (0.0392699197650774) | > loss_mel_ce: 3.264923572540283 (3.3238878907546288) | > loss: 0.8242866396903992 (0.8407894532522293) | > current_lr: 5e-06 | > step_time: 0.3167 (0.22983650348279241) | > loader_time: 0.0113 (0.009437944850877997) [1m --> TIME: 2024-12-19 14:00:26 -- STEP: 924/4076 -- GLOBAL_STEP: 5000[0m | > loss_text_ce: 0.036309681832790375 (0.03925634842785276) | > loss_mel_ce: 3.339580535888672 (3.3254444103736374) | > loss: 0.8439725637435913 (0.8411751901690578) | > current_lr: 5e-06 | > step_time: 0.1757 (0.22963563497964437) | > loader_time: 0.0087 (0.009413013468573116) [1m --> TIME: 2024-12-19 14:00:48 -- STEP: 974/4076 -- GLOBAL_STEP: 5050[0m | > loss_text_ce: 0.03190092742443085 (0.03926477044270022) | > loss_mel_ce: 3.3520610332489014 (3.3246931667445367) | > loss: 0.8459904789924622 (0.8409894846792828) | > current_lr: 5e-06 | > step_time: 0.2124 (0.23030323928386523) | > loader_time: 0.0082 (0.009402030302513808) [1m --> TIME: 2024-12-19 14:01:09 -- STEP: 1024/4076 -- GLOBAL_STEP: 5100[0m | > loss_text_ce: 0.04025161638855934 (0.039245459087396725) | > loss_mel_ce: 3.0921850204467773 (3.323781944345682) | > loss: 0.783109188079834 (0.8407568510156125) | > current_lr: 5e-06 | > step_time: 0.1743 (0.23036321136169136) | > loader_time: 0.0095 (0.009381246287375699) [1m --> TIME: 2024-12-19 14:01:29 -- STEP: 1074/4076 -- GLOBAL_STEP: 5150[0m | > loss_text_ce: 0.03867518901824951 (0.03921891216819955) | > loss_mel_ce: 3.4366884231567383 (3.320673923696663) | > loss: 0.8688409328460693 (0.8399732092350333) | > current_lr: 5e-06 | > step_time: 0.2872 (0.2302526817197462) | > loader_time: 0.0096 (0.00936202296997583) [1m --> TIME: 2024-12-19 14:01:50 -- STEP: 1124/4076 -- GLOBAL_STEP: 5200[0m | > loss_text_ce: 0.04245378449559212 (0.03919744333152881) | > loss_mel_ce: 3.1439809799194336 (3.3204106576077863) | > loss: 0.7966086864471436 (0.8399020254718026) | > current_lr: 5e-06 | > step_time: 0.1792 (0.23018984268568587) | > loader_time: 0.0084 (0.009352129973550718) [1m --> TIME: 2024-12-19 14:02:11 -- STEP: 1174/4076 -- GLOBAL_STEP: 5250[0m | > loss_text_ce: 0.03889482468366623 (0.039181313209324525) | > loss_mel_ce: 3.2966055870056152 (3.3210738125610986) | > loss: 0.8338751196861267 (0.8400637817646815) | > current_lr: 5e-06 | > step_time: 0.2501 (0.230218503706703) | > loader_time: 0.0092 (0.009347884374745262) [1m --> TIME: 2024-12-19 14:02:32 -- STEP: 1224/4076 -- GLOBAL_STEP: 5300[0m | > loss_text_ce: 0.03829943388700485 (0.03917794357193939) | > loss_mel_ce: 3.43062424659729 (3.323153932304943) | > loss: 0.8672308921813965 (0.840582969190638) | > current_lr: 5e-06 | > step_time: 0.235 (0.23011005626005285) | > loader_time: 0.0091 (0.009348782448986785) [1m --> TIME: 2024-12-19 14:02:53 -- STEP: 1274/4076 -- GLOBAL_STEP: 5350[0m | > loss_text_ce: 0.03396870195865631 (0.03915544094304367) | > loss_mel_ce: 3.256164312362671 (3.3248695745198655) | > loss: 0.8225332498550415 (0.8410062540082767) | > current_lr: 5e-06 | > step_time: 0.1875 (0.23017022542339666) | > loader_time: 0.0089 (0.009336058535792958) [1m --> TIME: 2024-12-19 14:03:14 -- STEP: 1324/4076 -- GLOBAL_STEP: 5400[0m | > loss_text_ce: 0.03431260213255882 (0.0391390460805005) | > loss_mel_ce: 2.928257703781128 (3.3241406850224173) | > loss: 0.7406425476074219 (0.8408199328130106) | > current_lr: 5e-06 | > step_time: 0.2112 (0.23005188555875933) | > loader_time: 0.0092 (0.009322349038368994) [1m --> TIME: 2024-12-19 14:03:35 -- STEP: 1374/4076 -- GLOBAL_STEP: 5450[0m | > loss_text_ce: 0.03799351677298546 (0.0391271560950397) | > loss_mel_ce: 3.6438677310943604 (3.322875474738242) | > loss: 0.9204652905464172 (0.8405006576486723) | > current_lr: 5e-06 | > step_time: 0.3584 (0.23012728878504324) | > loader_time: 0.0141 (0.009341342653943645) [1m --> TIME: 2024-12-19 14:03:56 -- STEP: 1424/4076 -- GLOBAL_STEP: 5500[0m | > loss_text_ce: 0.03438536077737808 (0.03910849890619347) | > loss_mel_ce: 3.294689178466797 (3.323252480351523) | > loss: 0.8322686553001404 (0.8405902448078889) | > current_lr: 5e-06 | > step_time: 0.1771 (0.22989485956979602) | > loader_time: 0.0097 (0.009355739931042287) [1m --> TIME: 2024-12-19 14:04:17 -- STEP: 1474/4076 -- GLOBAL_STEP: 5550[0m | > loss_text_ce: 0.03855245187878609 (0.03910813793246832) | > loss_mel_ce: 2.9745004177093506 (3.3206465004904135) | > loss: 0.7532632350921631 (0.8399386597213461) | > current_lr: 5e-06 | > step_time: 0.1839 (0.22992518491628536) | > loader_time: 0.0112 (0.00937807737891354) [1m --> TIME: 2024-12-19 14:04:38 -- STEP: 1524/4076 -- GLOBAL_STEP: 5600[0m | > loss_text_ce: 0.0376913957297802 (0.039089329369620424) | > loss_mel_ce: 3.239619731903076 (3.3201355076837413) | > loss: 0.8193277716636658 (0.8398062093207842) | > current_lr: 5e-06 | > step_time: 0.2378 (0.22994010967845366) | > loader_time: 0.0096 (0.009389307242365953) [1m --> TIME: 2024-12-19 14:04:59 -- STEP: 1574/4076 -- GLOBAL_STEP: 5650[0m | > loss_text_ce: 0.03880355507135391 (0.03907419974753566) | > loss_mel_ce: 3.270810842514038 (3.3199093781099234) | > loss: 0.8274036049842834 (0.8397458944720567) | > current_lr: 5e-06 | > step_time: 0.2801 (0.22985670254524357) | > loader_time: 0.0085 (0.00939595956814485) [1m --> TIME: 2024-12-19 14:05:20 -- STEP: 1624/4076 -- GLOBAL_STEP: 5700[0m | > loss_text_ce: 0.037070438265800476 (0.03907669345008702) | > loss_mel_ce: 3.1454849243164062 (3.32004900944644) | > loss: 0.7956388592720032 (0.8397814256363901) | > current_lr: 5e-06 | > step_time: 0.1743 (0.22989462631676585) | > loader_time: 0.0105 (0.009397825730845254) [1m --> TIME: 2024-12-19 14:05:41 -- STEP: 1674/4076 -- GLOBAL_STEP: 5750[0m | > loss_text_ce: 0.03599118813872337 (0.03906464231341833) | > loss_mel_ce: 3.4393882751464844 (3.3200970952228834) | > loss: 0.8688448667526245 (0.8397904343217647) | > current_lr: 5e-06 | > step_time: 0.2499 (0.23007250458821032) | > loader_time: 0.0086 (0.009398219833237294) [1m --> TIME: 2024-12-19 14:06:03 -- STEP: 1724/4076 -- GLOBAL_STEP: 5800[0m | > loss_text_ce: 0.03690173104405403 (0.03905196012302443) | > loss_mel_ce: 3.7447259426116943 (3.321078943127546) | > loss: 0.9454069137573242 (0.8400327256964973) | > current_lr: 5e-06 | > step_time: 0.2172 (0.23030433701805061) | > loader_time: 0.0089 (0.009399565770841806) [1m --> TIME: 2024-12-19 14:06:24 -- STEP: 1774/4076 -- GLOBAL_STEP: 5850[0m | > loss_text_ce: 0.040494292974472046 (0.039047208134811565) | > loss_mel_ce: 3.523160457611084 (3.3193986027458475) | > loss: 0.8909136652946472 (0.8396114526566416) | > current_lr: 5e-06 | > step_time: 0.2097 (0.23058018353772944) | > loader_time: 0.0088 (0.009390560454419125) [1m --> TIME: 2024-12-19 14:06:45 -- STEP: 1824/4076 -- GLOBAL_STEP: 5900[0m | > loss_text_ce: 0.039161887019872665 (0.03902432264090243) | > loss_mel_ce: 3.737309455871582 (3.319185330679542) | > loss: 0.9441178441047668 (0.8395524132473949) | > current_lr: 5e-06 | > step_time: 0.2743 (0.2304569425290091) | > loader_time: 0.0094 (0.00939115006149861) [1m --> TIME: 2024-12-19 14:07:06 -- STEP: 1874/4076 -- GLOBAL_STEP: 5950[0m | > loss_text_ce: 0.037073493003845215 (0.03901111039759451) | > loss_mel_ce: 3.3373024463653564 (3.318431382754315) | > loss: 0.843593955039978 (0.8393606230568861) | > current_lr: 5e-06 | > step_time: 0.1857 (0.23065578784861204) | > loader_time: 0.0105 (0.009389723759450679) [1m --> TIME: 2024-12-19 14:07:27 -- STEP: 1924/4076 -- GLOBAL_STEP: 6000[0m | > loss_text_ce: 0.03495389223098755 (0.03900528858317039) | > loss_mel_ce: 2.9385387897491455 (3.3188702300531703) | > loss: 0.7433731555938721 (0.8394688794431964) | > current_lr: 5e-06 | > step_time: 0.1772 (0.2305022489256274) | > loader_time: 0.0106 (0.009387769099332687) [1m --> TIME: 2024-12-19 14:07:47 -- STEP: 1974/4076 -- GLOBAL_STEP: 6050[0m | > loss_text_ce: 0.03697650879621506 (0.03899361377557425) | > loss_mel_ce: 3.247506856918335 (3.317371026481659) | > loss: 0.8211208581924438 (0.8390911597732111) | > current_lr: 5e-06 | > step_time: 0.2802 (0.23016252126374867) | > loader_time: 0.0104 (0.009387972869409254) [1m --> TIME: 2024-12-19 14:08:08 -- STEP: 2024/4076 -- GLOBAL_STEP: 6100[0m | > loss_text_ce: 0.036816276609897614 (0.038992270970856995) | > loss_mel_ce: 3.206068754196167 (3.316217747366004) | > loss: 0.8107212781906128 (0.8388025042685595) | > current_lr: 5e-06 | > step_time: 0.1909 (0.23012792675391489) | > loader_time: 0.0087 (0.009394942536184439) [1m --> TIME: 2024-12-19 14:08:30 -- STEP: 2074/4076 -- GLOBAL_STEP: 6150[0m | > loss_text_ce: 0.03885103762149811 (0.03898139103974077) | > loss_mel_ce: 3.264472723007202 (3.3159218369017687) | > loss: 0.8258309364318848 (0.8387258066620638) | > current_lr: 5e-06 | > step_time: 0.3214 (0.2303593238041509) | > loader_time: 0.0103 (0.00939757729680237) [1m --> TIME: 2024-12-19 14:08:51 -- STEP: 2124/4076 -- GLOBAL_STEP: 6200[0m | > loss_text_ce: 0.03482377529144287 (0.0389733663435654) | > loss_mel_ce: 3.2355008125305176 (3.3153095149949) | > loss: 0.8175811767578125 (0.8385707199461266) | > current_lr: 5e-06 | > step_time: 0.1905 (0.23032723229708196) | > loader_time: 0.0087 (0.0094024571546278) [1m --> TIME: 2024-12-19 14:09:11 -- STEP: 2174/4076 -- GLOBAL_STEP: 6250[0m | > loss_text_ce: 0.03752201050519943 (0.038957826767490385) | > loss_mel_ce: 3.3015592098236084 (3.314692115937226) | > loss: 0.8347703218460083 (0.8384124851873849) | > current_lr: 5e-06 | > step_time: 0.185 (0.23006287625654095) | > loader_time: 0.0102 (0.009411892105706152) [1m --> TIME: 2024-12-19 14:09:31 -- STEP: 2224/4076 -- GLOBAL_STEP: 6300[0m | > loss_text_ce: 0.03684575483202934 (0.038943166090778567) | > loss_mel_ce: 3.4237754344940186 (3.3147395893180964) | > loss: 0.8651552796363831 (0.8384206883019684) | > current_lr: 5e-06 | > step_time: 0.1746 (0.2298510367278573) | > loader_time: 0.0099 (0.009414339880291507) [1m --> TIME: 2024-12-19 14:09:53 -- STEP: 2274/4076 -- GLOBAL_STEP: 6350[0m | > loss_text_ce: 0.03701981529593468 (0.03895543669837818) | > loss_mel_ce: 3.2237424850463867 (3.314271894377788) | > loss: 0.8151905536651611 (0.8383068322558732) | > current_lr: 5e-06 | > step_time: 0.2931 (0.2301359852888969) | > loader_time: 0.0114 (0.009425580449133662) [1m --> TIME: 2024-12-19 14:10:15 -- STEP: 2324/4076 -- GLOBAL_STEP: 6400[0m | > loss_text_ce: 0.04045858234167099 (0.03894453407738125) | > loss_mel_ce: 3.4640438556671143 (3.313915929059768) | > loss: 0.8761256337165833 (0.8382151153534914) | > current_lr: 5e-06 | > step_time: 0.2712 (0.23024694572832835) | > loader_time: 0.0096 (0.009431487209004911) [1m --> TIME: 2024-12-19 14:10:35 -- STEP: 2374/4076 -- GLOBAL_STEP: 6450[0m | > loss_text_ce: 0.04096031188964844 (0.03892964210828866) | > loss_mel_ce: 3.151632070541382 (3.313605396006421) | > loss: 0.7981480956077576 (0.8381337591250001) | > current_lr: 5e-06 | > step_time: 0.2335 (0.23014575991192693) | > loader_time: 0.0092 (0.009430734570928247) [1m --> TIME: 2024-12-19 14:10:56 -- STEP: 2424/4076 -- GLOBAL_STEP: 6500[0m | > loss_text_ce: 0.03613778203725815 (0.038903090782408244) | > loss_mel_ce: 3.2172017097473145 (3.312246534690603) | > loss: 0.8133348822593689 (0.8377874059350582) | > current_lr: 5e-06 | > step_time: 0.2446 (0.23010171925274067) | > loader_time: 0.0095 (0.009435302255177264) [1m --> TIME: 2024-12-19 14:11:17 -- STEP: 2474/4076 -- GLOBAL_STEP: 6550[0m | > loss_text_ce: 0.0397186279296875 (0.03888559217881732) | > loss_mel_ce: 3.5199472904205322 (3.3110874534713326) | > loss: 0.8899164795875549 (0.8374932609504527) | > current_lr: 5e-06 | > step_time: 0.2821 (0.23011778898555238) | > loader_time: 0.0099 (0.0094458330226889) [1m --> TIME: 2024-12-19 14:11:38 -- STEP: 2524/4076 -- GLOBAL_STEP: 6600[0m | > loss_text_ce: 0.041924260556697845 (0.038865462406767234) | > loss_mel_ce: 3.3005895614624023 (3.3110473725572818) | > loss: 0.8356284499168396 (0.8374782083246111) | > current_lr: 5e-06 | > step_time: 0.2426 (0.23017024512147383) | > loader_time: 0.0089 (0.009434506462417586) [1m --> TIME: 2024-12-19 14:11:59 -- STEP: 2574/4076 -- GLOBAL_STEP: 6650[0m | > loss_text_ce: 0.039635222405195236 (0.038854498884379425) | > loss_mel_ce: 2.9825551509857178 (3.3103650403559994) | > loss: 0.7555475831031799 (0.8373048844361549) | > current_lr: 5e-06 | > step_time: 0.21 (0.23005122201055553) | > loader_time: 0.0086 (0.009429296691378793) [1m --> TIME: 2024-12-19 14:12:20 -- STEP: 2624/4076 -- GLOBAL_STEP: 6700[0m | > loss_text_ce: 0.037257593125104904 (0.03885407571712078) | > loss_mel_ce: 3.2866852283477783 (3.310131509220455) | > loss: 0.8309857249259949 (0.8372463958973931) | > current_lr: 5e-06 | > step_time: 0.1746 (0.2300235093003367) | > loader_time: 0.009 (0.009422446260365052) [1m --> TIME: 2024-12-19 14:12:40 -- STEP: 2674/4076 -- GLOBAL_STEP: 6750[0m | > loss_text_ce: 0.03755667805671692 (0.03884931231821736) | > loss_mel_ce: 3.118861198425293 (3.3100739880649566) | > loss: 0.7891044616699219 (0.8372308248535594) | > current_lr: 5e-06 | > step_time: 0.2125 (0.22993424808970125) | > loader_time: 0.0094 (0.009411899742120767) [1m --> TIME: 2024-12-19 14:13:01 -- STEP: 2724/4076 -- GLOBAL_STEP: 6800[0m | > loss_text_ce: 0.036625705659389496 (0.03884251211618084) | > loss_mel_ce: 3.6932904720306396 (3.3092567028803272) | > loss: 0.9324790239334106 (0.8370248035756155) | > current_lr: 5e-06 | > step_time: 0.1563 (0.22985126653606855) | > loader_time: 0.0088 (0.009409845050386053) [1m --> TIME: 2024-12-19 14:13:22 -- STEP: 2774/4076 -- GLOBAL_STEP: 6850[0m | > loss_text_ce: 0.040104255080223083 (0.03882906363072406) | > loss_mel_ce: 3.1205525398254395 (3.3086294637107985) | > loss: 0.7901641726493835 (0.8368646316633179) | > current_lr: 5e-06 | > step_time: 0.2361 (0.22987414119736968) | > loader_time: 0.0127 (0.00939723434599511) [1m --> TIME: 2024-12-19 14:13:43 -- STEP: 2824/4076 -- GLOBAL_STEP: 6900[0m | > loss_text_ce: 0.040011804550886154 (0.03881936230322107) | > loss_mel_ce: 3.2451071739196777 (3.30826476993034) | > loss: 0.821279764175415 (0.8367710328811292) | > current_lr: 5e-06 | > step_time: 0.1749 (0.22991606989933486) | > loader_time: 0.0083 (0.009386808777665972) [1m --> TIME: 2024-12-19 14:14:04 -- STEP: 2874/4076 -- GLOBAL_STEP: 6950[0m | > loss_text_ce: 0.03625376895070076 (0.0388107668062726) | > loss_mel_ce: 3.16796875 (3.3077854935958637) | > loss: 0.8010556101799011 (0.8366490649156634) | > current_lr: 5e-06 | > step_time: 0.1896 (0.22992051170365058) | > loader_time: 0.0084 (0.00937733869214149) [1m --> TIME: 2024-12-19 14:14:26 -- STEP: 2924/4076 -- GLOBAL_STEP: 7000[0m | > loss_text_ce: 0.03888561949133873 (0.03878855638183077) | > loss_mel_ce: 3.0603384971618652 (3.3073129317992036) | > loss: 0.774806022644043 (0.8365253719275696) | > current_lr: 5e-06 | > step_time: 0.2388 (0.23008767646901757) | > loader_time: 0.0092 (0.009366906944932448) [1m --> TIME: 2024-12-19 14:14:47 -- STEP: 2974/4076 -- GLOBAL_STEP: 7050[0m | > loss_text_ce: 0.036652062088251114 (0.03877386291496956) | > loss_mel_ce: 3.458688259124756 (3.30715323272133) | > loss: 0.8738350868225098 (0.8364817737611095) | > current_lr: 5e-06 | > step_time: 0.2826 (0.2302224221611152) | > loader_time: 0.0107 (0.00936809496825065) [1m --> TIME: 2024-12-19 14:15:08 -- STEP: 3024/4076 -- GLOBAL_STEP: 7100[0m | > loss_text_ce: 0.03968740627169609 (0.03876566821795511) | > loss_mel_ce: 3.20908522605896 (3.3066343646989296) | > loss: 0.8121931552886963 (0.8363500080766189) | > current_lr: 5e-06 | > step_time: 0.2351 (0.23024475952935605) | > loader_time: 0.0116 (0.009368915879537204) [1m --> TIME: 2024-12-19 14:15:29 -- STEP: 3074/4076 -- GLOBAL_STEP: 7150[0m | > loss_text_ce: 0.04084763303399086 (0.03876308641210441) | > loss_mel_ce: 3.6716082096099854 (3.306378364407869) | > loss: 0.9281139373779297 (0.8362853625182141) | > current_lr: 5e-06 | > step_time: 0.2935 (0.23013575951930534) | > loader_time: 0.0088 (0.009365856065793566) [1m --> TIME: 2024-12-19 14:15:50 -- STEP: 3124/4076 -- GLOBAL_STEP: 7200[0m | > loss_text_ce: 0.03569769859313965 (0.03876609705858381) | > loss_mel_ce: 3.38797664642334 (3.3059235884499154) | > loss: 0.8559185862541199 (0.8361724211754478) | > current_lr: 5e-06 | > step_time: 0.2092 (0.23015635846976898) | > loader_time: 0.009 (0.009358436594241414) [1m --> TIME: 2024-12-19 14:16:10 -- STEP: 3174/4076 -- GLOBAL_STEP: 7250[0m | > loss_text_ce: 0.04487138241529465 (0.0387576448313878) | > loss_mel_ce: 3.6283118724823 (3.305249407070571) | > loss: 0.9182958006858826 (0.836001762745887) | > current_lr: 5e-06 | > step_time: 0.2755 (0.23007375298616048) | > loader_time: 0.0088 (0.009346352332631374) [1m --> TIME: 2024-12-19 14:16:31 -- STEP: 3224/4076 -- GLOBAL_STEP: 7300[0m | > loss_text_ce: 0.03710361570119858 (0.03874688131547539) | > loss_mel_ce: 3.0724148750305176 (3.3042863807074783) | > loss: 0.7773796319961548 (0.8357583152744968) | > current_lr: 5e-06 | > step_time: 0.1778 (0.23010299711606055) | > loader_time: 0.0082 (0.00933486792644851) [1m --> TIME: 2024-12-19 14:16:53 -- STEP: 3274/4076 -- GLOBAL_STEP: 7350[0m | > loss_text_ce: 0.03665923699736595 (0.038736438438428306) | > loss_mel_ce: 3.2828242778778076 (3.303383185795042) | > loss: 0.829870879650116 (0.835529905852561) | > current_lr: 5e-06 | > step_time: 0.2867 (0.23031770585875147) | > loader_time: 0.0084 (0.009326353623652944) [1m --> TIME: 2024-12-19 14:17:14 -- STEP: 3324/4076 -- GLOBAL_STEP: 7400[0m | > loss_text_ce: 0.039389993995428085 (0.03872558330471496) | > loss_mel_ce: 3.4569060802459717 (3.3025098060729556) | > loss: 0.8740740418434143 (0.8353088471038823) | > current_lr: 5e-06 | > step_time: 0.1927 (0.23026025273739653) | > loader_time: 0.0088 (0.009315487781873539) [1m --> TIME: 2024-12-19 14:17:35 -- STEP: 3374/4076 -- GLOBAL_STEP: 7450[0m | > loss_text_ce: 0.040152668952941895 (0.03872154173057551) | > loss_mel_ce: 3.3878962993621826 (3.301535397291042) | > loss: 0.8570122718811035 (0.8350642345752957) | > current_lr: 5e-06 | > step_time: 0.2312 (0.23024407857681217) | > loader_time: 0.008 (0.009306375840463515) [1m --> TIME: 2024-12-19 14:17:56 -- STEP: 3424/4076 -- GLOBAL_STEP: 7500[0m | > loss_text_ce: 0.034645043313503265 (0.0387142745023985) | > loss_mel_ce: 3.459360122680664 (3.3007813110669084) | > loss: 0.8735013008117676 (0.8348738961966241) | > current_lr: 5e-06 | > step_time: 0.2962 (0.23018580765646202) | > loader_time: 0.0093 (0.009297039394623782) [1m --> TIME: 2024-12-19 14:18:17 -- STEP: 3474/4076 -- GLOBAL_STEP: 7550[0m | > loss_text_ce: 0.03600556403398514 (0.03870749197606475) | > loss_mel_ce: 3.3643686771392822 (3.3002355305298896) | > loss: 0.8500935435295105 (0.8347357554413771) | > current_lr: 5e-06 | > step_time: 0.2592 (0.2301552838543468) | > loader_time: 0.0087 (0.009297661213059511) [1m --> TIME: 2024-12-19 14:18:37 -- STEP: 3524/4076 -- GLOBAL_STEP: 7600[0m | > loss_text_ce: 0.03439847379922867 (0.038697742040942465) | > loss_mel_ce: 3.0093090534210205 (3.299799289321791) | > loss: 0.7609269022941589 (0.8346242576194813) | > current_lr: 5e-06 | > step_time: 0.1512 (0.23004285052469456) | > loader_time: 0.0079 (0.009293661282633557) [1m --> TIME: 2024-12-19 14:18:58 -- STEP: 3574/4076 -- GLOBAL_STEP: 7650[0m | > loss_text_ce: 0.038291677832603455 (0.038688748800281746) | > loss_mel_ce: 3.2690393924713135 (3.2995102711076103) | > loss: 0.8268327713012695 (0.834549754791308) | > current_lr: 5e-06 | > step_time: 0.2507 (0.230118008785963) | > loader_time: 0.0092 (0.009287997871787452) [1m --> TIME: 2024-12-19 14:19:19 -- STEP: 3624/4076 -- GLOBAL_STEP: 7700[0m | > loss_text_ce: 0.036876462399959564 (0.038675766636126124) | > loss_mel_ce: 3.3673810958862305 (3.2988182216816915) | > loss: 0.8510643839836121 (0.8343734968840157) | > current_lr: 5e-06 | > step_time: 0.1896 (0.23011305316394534) | > loader_time: 0.0085 (0.009284250915182096) [1m --> TIME: 2024-12-19 14:19:40 -- STEP: 3674/4076 -- GLOBAL_STEP: 7750[0m | > loss_text_ce: 0.0370124913752079 (0.03867205495758019) | > loss_mel_ce: 3.1429145336151123 (3.298766251874476) | > loss: 0.7949817776679993 (0.8343595765372888) | > current_lr: 5e-06 | > step_time: 0.1868 (0.23008766729804447) | > loader_time: 0.0086 (0.00927671538660822) [1m --> TIME: 2024-12-19 14:20:01 -- STEP: 3724/4076 -- GLOBAL_STEP: 7800[0m | > loss_text_ce: 0.03494982421398163 (0.03866389901632349) | > loss_mel_ce: 3.40122127532959 (3.2981170823059327) | > loss: 0.859042763710022 (0.8341952451733848) | > current_lr: 5e-06 | > step_time: 0.2719 (0.23007082081250813) | > loader_time: 0.0107 (0.009278913509704896) [1m --> TIME: 2024-12-19 14:20:22 -- STEP: 3774/4076 -- GLOBAL_STEP: 7850[0m | > loss_text_ce: 0.0379205159842968 (0.03865388946022363) | > loss_mel_ce: 3.222552537918091 (3.297907204241366) | > loss: 0.8151182532310486 (0.8341402733016409) | > current_lr: 5e-06 | > step_time: 0.2605 (0.23014385806358112) | > loader_time: 0.011 (0.009280543827669432) [1m --> TIME: 2024-12-19 14:20:44 -- STEP: 3824/4076 -- GLOBAL_STEP: 7900[0m | > loss_text_ce: 0.03811342269182205 (0.038648295952260464) | > loss_mel_ce: 3.1469080448150635 (3.2975058556226506) | > loss: 0.796255350112915 (0.834038537806173) | > current_lr: 5e-06 | > step_time: 0.1732 (0.23021131878617423) | > loader_time: 0.0089 (0.009278341254928633) [1m --> TIME: 2024-12-19 14:21:05 -- STEP: 3874/4076 -- GLOBAL_STEP: 7950[0m | > loss_text_ce: 0.04330223426222801 (0.038644813210787575) | > loss_mel_ce: 3.3489575386047363 (3.2968622473754645) | > loss: 0.8480649590492249 (0.8338767650758847) | > current_lr: 5e-06 | > step_time: 0.1873 (0.230196054411353) | > loader_time: 0.0098 (0.009275351287042238) [1m --> TIME: 2024-12-19 14:21:25 -- STEP: 3924/4076 -- GLOBAL_STEP: 8000[0m | > loss_text_ce: 0.04132049158215523 (0.03863963958589886) | > loss_mel_ce: 3.5110373497009277 (3.295612557218709) | > loss: 0.8880894780158997 (0.8335630491634152) | > current_lr: 5e-06 | > step_time: 0.1572 (0.23001545995747283) | > loader_time: 0.0096 (0.009272890717975966) [1m --> TIME: 2024-12-19 14:21:46 -- STEP: 3974/4076 -- GLOBAL_STEP: 8050[0m | > loss_text_ce: 0.03665989264845848 (0.038639812352005476) | > loss_mel_ce: 3.549016237258911 (3.2950478329032165) | > loss: 0.8964190483093262 (0.8334219113299761) | > current_lr: 5e-06 | > step_time: 0.2328 (0.23004410653241514) | > loader_time: 0.0082 (0.009263855341876289) [1m --> TIME: 2024-12-19 14:22:07 -- STEP: 4024/4076 -- GLOBAL_STEP: 8100[0m | > loss_text_ce: 0.03860511630773544 (0.03863238373115866) | > loss_mel_ce: 3.422027349472046 (3.2944567072581106) | > loss: 0.8651581406593323 (0.8332722727394486) | > current_lr: 5e-06 | > step_time: 0.1747 (0.23000665847397236) | > loader_time: 0.0084 (0.009257288327510988) [1m --> TIME: 2024-12-19 14:22:27 -- STEP: 4074/4076 -- GLOBAL_STEP: 8150[0m | > loss_text_ce: 0.039389029145240784 (0.038623422618350035) | > loss_mel_ce: 2.9128599166870117 (3.29402234083774) | > loss: 0.7380622625350952 (0.8331614408384194) | > current_lr: 5e-06 | > step_time: 0.2597 (0.22997630695882362) | > loader_time: 0.0084 (0.009251623862973029) [1m > EVALUATION [0m [1m--> EVAL PERFORMANCE[0m | > avg_loader_time:[91m 0.47067922812241775 [0m(+0.0010625490775474922) | > avg_loss_text_ce:[92m 0.03682947287765833 [0m(-0.0013035371039922528) | > avg_loss_mel_ce:[92m 3.260605060137235 [0m(-0.06250026592841529) | > avg_loss:[92m 3.2974345225554247 [0m(-0.06380381950965308) [4m[1m > EPOCH: 2/8[0m --> /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb [1m > TRAINING (2024-12-19 14:23:11) [0m [1m --> TIME: 2024-12-19 14:23:32 -- STEP: 48/4076 -- GLOBAL_STEP: 8200[0m | > loss_text_ce: 0.03626027703285217 (0.03817165914612511) | > loss_mel_ce: 3.4999470710754395 (3.2366769214471183) | > loss: 0.8840518593788147 (0.8187121426065763) | > current_lr: 5e-06 | > step_time: 0.2327 (0.22930882374445596) | > loader_time: 0.009 (0.00958996017773946) [1m --> TIME: 2024-12-19 14:23:52 -- STEP: 98/4076 -- GLOBAL_STEP: 8250[0m | > loss_text_ce: 0.03861541301012039 (0.038247764566723186) | > loss_mel_ce: 3.2011284828186035 (3.256605471883501) | > loss: 0.809935986995697 (0.8237133068697793) | > current_lr: 5e-06 | > step_time: 0.1882 (0.2272269409530017) | > loader_time: 0.0087 (0.009361352239336283) [1m --> TIME: 2024-12-19 14:24:13 -- STEP: 148/4076 -- GLOBAL_STEP: 8300[0m | > loss_text_ce: 0.037651222199201584 (0.03813263059065147) | > loss_mel_ce: 3.255227565765381 (3.2644569793262996) | > loss: 0.8232197165489197 (0.8256474008431306) | > current_lr: 5e-06 | > step_time: 0.2324 (0.22728404805466934) | > loader_time: 0.0091 (0.009346330488050303) [1m --> TIME: 2024-12-19 14:24:34 -- STEP: 198/4076 -- GLOBAL_STEP: 8350[0m | > loss_text_ce: 0.03638994321227074 (0.038069095236785463) | > loss_mel_ce: 3.381298303604126 (3.2500373980011603) | > loss: 0.8544220328330994 (0.8220266215126923) | > current_lr: 5e-06 | > step_time: 0.2143 (0.226507402429677) | > loader_time: 0.009 (0.009322668566848286) [1m --> TIME: 2024-12-19 14:24:55 -- STEP: 248/4076 -- GLOBAL_STEP: 8400[0m | > loss_text_ce: 0.035118091851472855 (0.038032472013465796) | > loss_mel_ce: 3.3643076419830322 (3.262295490311038) | > loss: 0.849856436252594 (0.8250819896498034) | > current_lr: 5e-06 | > step_time: 0.2096 (0.22836654416976437) | > loader_time: 0.0086 (0.009277498529803363) [1m --> TIME: 2024-12-19 14:25:16 -- STEP: 298/4076 -- GLOBAL_STEP: 8450[0m | > loss_text_ce: 0.03680052608251572 (0.038106887382548894) | > loss_mel_ce: 3.1227169036865234 (3.2678011823820587) | > loss: 0.7898793816566467 (0.8264770165785849) | > current_lr: 5e-06 | > step_time: 0.2497 (0.22910057858332702) | > loader_time: 0.0101 (0.009250908089964182) [1m --> TIME: 2024-12-19 14:25:38 -- STEP: 348/4076 -- GLOBAL_STEP: 8500[0m | > loss_text_ce: 0.04384566843509674 (0.03806917480696205) | > loss_mel_ce: 3.7913811206817627 (3.2632220401161014) | > loss: 0.9588066935539246 (0.825322802553232) | > current_lr: 5e-06 | > step_time: 0.2657 (0.22919749528512195) | > loader_time: 0.0092 (0.009259792580001654) [1m --> TIME: 2024-12-19 14:25:58 -- STEP: 398/4076 -- GLOBAL_STEP: 8550[0m | > loss_text_ce: 0.04202587530016899 (0.038190865084229406) | > loss_mel_ce: 3.3795244693756104 (3.2590929611244386) | > loss: 0.8553875684738159 (0.8243209554921446) | > current_lr: 5e-06 | > step_time: 0.2086 (0.22850807947130067) | > loader_time: 0.0099 (0.009179580750776902) [1m --> TIME: 2024-12-19 14:26:19 -- STEP: 448/4076 -- GLOBAL_STEP: 8600[0m | > loss_text_ce: 0.04006893187761307 (0.03818758239503951) | > loss_mel_ce: 2.9159774780273438 (3.2553362372730454) | > loss: 0.7390115857124329 (0.8233809536322954) | > current_lr: 5e-06 | > step_time: 0.2099 (0.22814179531165538) | > loader_time: 0.0079 (0.009172318769352785) [1m --> TIME: 2024-12-19 14:26:40 -- STEP: 498/4076 -- GLOBAL_STEP: 8650[0m | > loss_text_ce: 0.039874445647001266 (0.03819130592975752) | > loss_mel_ce: 3.3970465660095215 (3.25273227835276) | > loss: 0.8592302799224854 (0.8227308952664757) | > current_lr: 5e-06 | > step_time: 0.2122 (0.22831180440374174) | > loader_time: 0.0095 (0.009175804245424087) [1m --> TIME: 2024-12-19 14:27:01 -- STEP: 548/4076 -- GLOBAL_STEP: 8700[0m | > loss_text_ce: 0.03832070529460907 (0.038175234407947886) | > loss_mel_ce: 3.8127999305725098 (3.2572610369563963) | > loss: 0.9627801775932312 (0.823859067185082) | > current_lr: 5e-06 | > step_time: 0.1769 (0.22947519234497188) | > loader_time: 0.0089 (0.009170615760079272) [1m --> TIME: 2024-12-19 14:27:22 -- STEP: 598/4076 -- GLOBAL_STEP: 8750[0m | > loss_text_ce: 0.03852162882685661 (0.038204268069570294) | > loss_mel_ce: 3.2659764289855957 (3.257161345370237) | > loss: 0.8261244893074036 (0.8238414024628924) | > current_lr: 5e-06 | > step_time: 0.1699 (0.22912974620742546) | > loader_time: 0.008 (0.00918603381983014) [1m --> TIME: 2024-12-19 14:27:44 -- STEP: 648/4076 -- GLOBAL_STEP: 8800[0m | > loss_text_ce: 0.03248317167162895 (0.038136429578801756) | > loss_mel_ce: 3.070723295211792 (3.256076639817082) | > loss: 0.7758015990257263 (0.8235532662566797) | > current_lr: 5e-06 | > step_time: 0.215 (0.2297892695591774) | > loader_time: 0.0106 (0.009207651203061325) [1m --> TIME: 2024-12-19 14:28:05 -- STEP: 698/4076 -- GLOBAL_STEP: 8850[0m | > loss_text_ce: 0.03605649992823601 (0.038153813132498555) | > loss_mel_ce: 3.4266979694366455 (3.2536958742961475) | > loss: 0.8656886219978333 (0.8229624208884799) | > current_lr: 5e-06 | > step_time: 0.2326 (0.23024907221425234) | > loader_time: 0.0097 (0.009204508581953952) [1m --> TIME: 2024-12-19 14:28:25 -- STEP: 748/4076 -- GLOBAL_STEP: 8900[0m | > loss_text_ce: 0.03816567361354828 (0.03814516498154422) | > loss_mel_ce: 2.8918066024780273 (3.251555550863396) | > loss: 0.7324930429458618 (0.8224251781874162) | > current_lr: 5e-06 | > step_time: 0.1573 (0.22961644119119903) | > loader_time: 0.0089 (0.009228631774371953) [1m --> TIME: 2024-12-19 14:28:46 -- STEP: 798/4076 -- GLOBAL_STEP: 8950[0m | > loss_text_ce: 0.04386631026864052 (0.03813037254093665) | > loss_mel_ce: 3.409463405609131 (3.253724728610579) | > loss: 0.8633324503898621 (0.8229637746524093) | > current_lr: 5e-06 | > step_time: 0.1882 (0.22944259195399466) | > loader_time: 0.0092 (0.009218732814741011) [1m --> TIME: 2024-12-19 14:29:07 -- STEP: 848/4076 -- GLOBAL_STEP: 9000[0m | > loss_text_ce: 0.03775245323777199 (0.038112779343412936) | > loss_mel_ce: 2.927438259124756 (3.2551048895660415) | > loss: 0.7412976622581482 (0.8233044166469349) | > current_lr: 5e-06 | > step_time: 0.1749 (0.22934882539623191) | > loader_time: 0.0092 (0.009223566302713346) [1m --> TIME: 2024-12-19 14:29:28 -- STEP: 898/4076 -- GLOBAL_STEP: 9050[0m | > loss_text_ce: 0.040499839931726456 (0.03810853153292951) | > loss_mel_ce: 3.43332576751709 (3.2545890032846816) | > loss: 0.8684564232826233 (0.8231743830349504) | > current_lr: 5e-06 | > step_time: 0.281 (0.2296806597762756) | > loader_time: 0.0092 (0.00924929930000899) [1m --> TIME: 2024-12-19 14:29:49 -- STEP: 948/4076 -- GLOBAL_STEP: 9100[0m | > loss_text_ce: 0.03842298313975334 (0.03808965532989385) | > loss_mel_ce: 3.485396385192871 (3.253466569170165) | > loss: 0.8809548616409302 (0.8228890555065895) | > current_lr: 5e-06 | > step_time: 0.2675 (0.2296261792444479) | > loader_time: 0.01 (0.009258600981426633) [1m --> TIME: 2024-12-19 14:30:10 -- STEP: 998/4076 -- GLOBAL_STEP: 9150[0m | > loss_text_ce: 0.0368879996240139 (0.03807816837535419) | > loss_mel_ce: 3.1710402965545654 (3.2520260395172347) | > loss: 0.801982045173645 (0.8225260513698409) | > current_lr: 5e-06 | > step_time: 0.1877 (0.22933819012078113) | > loader_time: 0.0095 (0.00927316043563261) [1m --> TIME: 2024-12-19 14:30:31 -- STEP: 1048/4076 -- GLOBAL_STEP: 9200[0m | > loss_text_ce: 0.03853333741426468 (0.038086988481529506) | > loss_mel_ce: 3.121776819229126 (3.252522320692774) | > loss: 0.7900775671005249 (0.8226523268541307) | > current_lr: 5e-06 | > step_time: 0.2439 (0.2293087683106197) | > loader_time: 0.0093 (0.009273340911355631) [1m --> TIME: 2024-12-19 14:30:52 -- STEP: 1098/4076 -- GLOBAL_STEP: 9250[0m | > loss_text_ce: 0.03937617689371109 (0.03808555419714567) | > loss_mel_ce: 2.797837734222412 (3.2533733846928463) | > loss: 0.7093034982681274 (0.8228647342470826) | > current_lr: 5e-06 | > step_time: 0.21 (0.2294309595243527) | > loader_time: 0.0086 (0.009293855430867061) [1m --> TIME: 2024-12-19 14:31:13 -- STEP: 1148/4076 -- GLOBAL_STEP: 9300[0m | > loss_text_ce: 0.03652878478169441 (0.03807773696622112) | > loss_mel_ce: 3.100968599319458 (3.2519051658982616) | > loss: 0.7843743562698364 (0.8224957251486466) | > current_lr: 5e-06 | > step_time: 0.2097 (0.22960682924616213) | > loader_time: 0.0105 (0.009299268913601328) [1m --> TIME: 2024-12-19 14:31:34 -- STEP: 1198/4076 -- GLOBAL_STEP: 9350[0m | > loss_text_ce: 0.03654756769537926 (0.038070077118533836) | > loss_mel_ce: 3.371950149536133 (3.251633623605581) | > loss: 0.8521244525909424 (0.8224259247266391) | > current_lr: 5e-06 | > step_time: 0.1844 (0.2293796081574811) | > loader_time: 0.008 (0.009278692864814453) [1m --> TIME: 2024-12-19 14:31:55 -- STEP: 1248/4076 -- GLOBAL_STEP: 9400[0m | > loss_text_ce: 0.039463821798563004 (0.038059042866688056) | > loss_mel_ce: 2.8889827728271484 (3.2520843217006106) | > loss: 0.7321116328239441 (0.822535840746684) | > current_lr: 5e-06 | > step_time: 0.1536 (0.22930968705660257) | > loader_time: 0.0095 (0.009260608026614546) [1m --> TIME: 2024-12-19 14:32:15 -- STEP: 1298/4076 -- GLOBAL_STEP: 9450[0m | > loss_text_ce: 0.03803325444459915 (0.03807771450571591) | > loss_mel_ce: 3.4222302436828613 (3.251033571541586) | > loss: 0.86506587266922 (0.8222778210379126) | > current_lr: 5e-06 | > step_time: 0.2323 (0.22910972681177783) | > loader_time: 0.0089 (0.00924586478293584) [1m --> TIME: 2024-12-19 14:32:36 -- STEP: 1348/4076 -- GLOBAL_STEP: 9500[0m | > loss_text_ce: 0.03880969062447548 (0.03806728860556817) | > loss_mel_ce: 3.534621238708496 (3.249168638836384) | > loss: 0.8933577537536621 (0.821808981435589) | > current_lr: 5e-06 | > step_time: 0.1901 (0.22899072849432156) | > loader_time: 0.0079 (0.009255276235699293) [1m --> TIME: 2024-12-19 14:32:57 -- STEP: 1398/4076 -- GLOBAL_STEP: 9550[0m | > loss_text_ce: 0.03377637267112732 (0.038059279122769595) | > loss_mel_ce: 3.2540688514709473 (3.248174589249881) | > loss: 0.8219612836837769 (0.8215584665408974) | > current_lr: 5e-06 | > step_time: 0.2098 (0.22921586787069645) | > loader_time: 0.0099 (0.009249984280063695) [1m --> TIME: 2024-12-19 14:33:18 -- STEP: 1448/4076 -- GLOBAL_STEP: 9600[0m | > loss_text_ce: 0.03717360273003578 (0.03807497227780562) | > loss_mel_ce: 3.0647706985473633 (3.2499411919498984) | > loss: 0.7754860520362854 (0.8220040404664878) | > current_lr: 5e-06 | > step_time: 0.1554 (0.22925892966228295) | > loader_time: 0.0101 (0.009249011126671045) [1m --> TIME: 2024-12-19 14:33:39 -- STEP: 1498/4076 -- GLOBAL_STEP: 9650[0m | > loss_text_ce: 0.03742736205458641 (0.03806068673630424) | > loss_mel_ce: 3.090364933013916 (3.2491865823361215) | > loss: 0.7819480895996094 (0.8218118166812113) | > current_lr: 5e-06 | > step_time: 0.2744 (0.22919983157169357) | > loader_time: 0.0082 (0.009249736852098366) [1m --> TIME: 2024-12-19 14:34:00 -- STEP: 1548/4076 -- GLOBAL_STEP: 9700[0m | > loss_text_ce: 0.03884747996926308 (0.03805888822971234) | > loss_mel_ce: 3.2280354499816895 (3.2469159424767025) | > loss: 0.816720724105835 (0.8212437071387466) | > current_lr: 5e-06 | > step_time: 0.2162 (0.22916200456693192) | > loader_time: 0.0106 (0.00924115587574567) [1m --> TIME: 2024-12-19 14:34:21 -- STEP: 1598/4076 -- GLOBAL_STEP: 9750[0m | > loss_text_ce: 0.034627869725227356 (0.038037262034039426) | > loss_mel_ce: 2.930222749710083 (3.2464112015033106) | > loss: 0.7412126660346985 (0.8211121154070199) | > current_lr: 5e-06 | > step_time: 0.2323 (0.22915590107813943) | > loader_time: 0.0089 (0.009227956937758889) [1m --> TIME: 2024-12-19 14:34:42 -- STEP: 1648/4076 -- GLOBAL_STEP: 9800[0m | > loss_text_ce: 0.03453225642442703 (0.03803088534532961) | > loss_mel_ce: 3.120225191116333 (3.245264458569509) | > loss: 0.788689374923706 (0.820823835480271) | > current_lr: 5e-06 | > step_time: 0.1911 (0.22919456183331685) | > loader_time: 0.008 (0.00921517263338404) [1m --> TIME: 2024-12-19 14:35:03 -- STEP: 1698/4076 -- GLOBAL_STEP: 9850[0m | > loss_text_ce: 0.038504961878061295 (0.03803053987843231) | > loss_mel_ce: 3.5831878185272217 (3.2439263382283774) | > loss: 0.9054232239723206 (0.8204892191013543) | > current_lr: 5e-06 | > step_time: 0.2116 (0.2291747498708844) | > loader_time: 0.0087 (0.0091941415631729) [1m --> TIME: 2024-12-19 14:35:23 -- STEP: 1748/4076 -- GLOBAL_STEP: 9900[0m | > loss_text_ce: 0.038712527602910995 (0.038013998827165996) | > loss_mel_ce: 3.211672782897949 (3.242556487259113) | > loss: 0.812596321105957 (0.8201426211019948) | > current_lr: 5e-06 | > step_time: 0.1567 (0.22907317025710572) | > loader_time: 0.0088 (0.009195045284602956) [1m --> TIME: 2024-12-19 14:35:44 -- STEP: 1798/4076 -- GLOBAL_STEP: 9950[0m | > loss_text_ce: 0.03357231244444847 (0.037992556838399485) | > loss_mel_ce: 3.190661907196045 (3.2404241229058384) | > loss: 0.8060585260391235 (0.8196041695069153) | > current_lr: 5e-06 | > step_time: 0.1865 (0.22907663306617101) | > loader_time: 0.01 (0.009194343188182932) [1m --> TIME: 2024-12-19 14:36:05 -- STEP: 1848/4076 -- GLOBAL_STEP: 10000[0m | > loss_text_ce: 0.040814079344272614 (0.03798293292792552) | > loss_mel_ce: 2.912106990814209 (3.2394644467087543) | > loss: 0.738230288028717 (0.8193618445556401) | > current_lr: 5e-06 | > step_time: 0.1757 (0.2290990294574143) | > loader_time: 0.0095 (0.009194512000847692) [1m --> TIME: 2024-12-19 14:36:25 -- STEP: 1898/4076 -- GLOBAL_STEP: 10050[0m | > loss_text_ce: 0.040064696222543716 (0.03798876545078917) | > loss_mel_ce: 3.3870041370391846 (3.2390915200632424) | > loss: 0.8567672371864319 (0.8192700710387074) | > current_lr: 5e-06 | > step_time: 0.1842 (0.22885524121174947) | > loader_time: 0.0101 (0.009199113438830364) [1m --> TIME: 2024-12-19 14:36:47 -- STEP: 1948/4076 -- GLOBAL_STEP: 10100[0m | > loss_text_ce: 0.03625311702489853 (0.037974577596950373) | > loss_mel_ce: 3.1738827228546143 (3.2393395954578574) | > loss: 0.8025339841842651 (0.8193285428829017) | > current_lr: 5e-06 | > step_time: 0.243 (0.22888049758680057) | > loader_time: 0.0092 (0.009203471564659107) [1m --> TIME: 2024-12-19 14:37:07 -- STEP: 1998/4076 -- GLOBAL_STEP: 10150[0m | > loss_text_ce: 0.04008658230304718 (0.03796257692723867) | > loss_mel_ce: 3.074103355407715 (3.2378392615714477) | > loss: 0.778547465801239 (0.8189504592566638) | > current_lr: 5e-06 | > step_time: 0.2589 (0.228857275243994) | > loader_time: 0.0104 (0.00920230788630886) [1m --> TIME: 2024-12-19 14:37:29 -- STEP: 2048/4076 -- GLOBAL_STEP: 10200[0m | > loss_text_ce: 0.03986237198114395 (0.03796341956785907) | > loss_mel_ce: 3.257836103439331 (3.2374342832481497) | > loss: 0.8244246244430542 (0.8188494253263343) | > current_lr: 5e-06 | > step_time: 0.1908 (0.22901357105001807) | > loader_time: 0.0086 (0.009200493106618527) [1m --> TIME: 2024-12-19 14:37:50 -- STEP: 2098/4076 -- GLOBAL_STEP: 10250[0m | > loss_text_ce: 0.03745539113879204 (0.03796311015009282) | > loss_mel_ce: 3.1915695667266846 (3.237072292796992) | > loss: 0.8072562217712402 (0.818758850357894) | > current_lr: 5e-06 | > step_time: 0.2875 (0.22899967687714087) | > loader_time: 0.0088 (0.00920321579997715) [1m --> TIME: 2024-12-19 14:38:10 -- STEP: 2148/4076 -- GLOBAL_STEP: 10300[0m | > loss_text_ce: 0.04117700457572937 (0.03794894687349263) | > loss_mel_ce: 3.116699457168579 (3.2364910357047294) | > loss: 0.7894691228866577 (0.8186099952992113) | > current_lr: 5e-06 | > step_time: 0.158 (0.2289245493807162) | > loader_time: 0.0087 (0.009192587610063606) [1m --> TIME: 2024-12-19 14:38:32 -- STEP: 2198/4076 -- GLOBAL_STEP: 10350[0m | > loss_text_ce: 0.03716028481721878 (0.037946970510492256) | > loss_mel_ce: 3.4474058151245117 (3.236433055750125) | > loss: 0.8711415529251099 (0.8185950062348695) | > current_lr: 5e-06 | > step_time: 0.2291 (0.22916725139600563) | > loader_time: 0.009 (0.00919744239491262) [1m --> TIME: 2024-12-19 14:38:53 -- STEP: 2248/4076 -- GLOBAL_STEP: 10400[0m | > loss_text_ce: 0.029555901885032654 (0.037934206670574365) | > loss_mel_ce: 3.3595972061157227 (3.2364398124379203) | > loss: 0.8472882509231567 (0.8185935044840139) | > current_lr: 5e-06 | > step_time: 0.1518 (0.22918988631713427) | > loader_time: 0.0084 (0.009194805316653537) [1m --> TIME: 2024-12-19 14:39:15 -- STEP: 2298/4076 -- GLOBAL_STEP: 10450[0m | > loss_text_ce: 0.03822503238916397 (0.037926421417255704) | > loss_mel_ce: 3.072633743286133 (3.236394859169546) | > loss: 0.7777146697044373 (0.8185803198990768) | > current_lr: 5e-06 | > step_time: 0.2582 (0.22960290865238275) | > loader_time: 0.0084 (0.009200998551955525) [1m --> TIME: 2024-12-19 14:39:36 -- STEP: 2348/4076 -- GLOBAL_STEP: 10500[0m | > loss_text_ce: 0.03939785808324814 (0.037911807627177566) | > loss_mel_ce: 3.516494035720825 (3.2374444560257336) | > loss: 0.8889729976654053 (0.8188390657686494) | > current_lr: 5e-06 | > step_time: 0.2348 (0.22944405304309248) | > loader_time: 0.0089 (0.009201074276100429) [1m --> TIME: 2024-12-19 14:39:56 -- STEP: 2398/4076 -- GLOBAL_STEP: 10550[0m | > loss_text_ce: 0.04067816585302353 (0.03790959518781865) | > loss_mel_ce: 3.269991874694824 (3.236415215290224) | > loss: 0.8276675343513489 (0.8185812026088288) | > current_lr: 5e-06 | > step_time: 0.2805 (0.2294323688949317) | > loader_time: 0.0096 (0.009193361849462713) [1m --> TIME: 2024-12-19 14:40:18 -- STEP: 2448/4076 -- GLOBAL_STEP: 10600[0m | > loss_text_ce: 0.03675229847431183 (0.03790813737649318) | > loss_mel_ce: 3.209099531173706 (3.235852076629412) | > loss: 0.811462938785553 (0.8184400535146007) | > current_lr: 5e-06 | > step_time: 0.1927 (0.22957807000166447) | > loader_time: 0.0091 (0.009185440205281075) [1m --> TIME: 2024-12-19 14:40:38 -- STEP: 2498/4076 -- GLOBAL_STEP: 10650[0m | > loss_text_ce: 0.03533666953444481 (0.03789835072102377) | > loss_mel_ce: 3.4325642585754395 (3.2351006370625415) | > loss: 0.8669752478599548 (0.8182497469304173) | > current_lr: 5e-06 | > step_time: 0.2082 (0.22932946681976313) | > loader_time: 0.0102 (0.009187499936625317) [1m --> TIME: 2024-12-19 14:40:59 -- STEP: 2548/4076 -- GLOBAL_STEP: 10700[0m | > loss_text_ce: 0.0404665470123291 (0.03789822046690307) | > loss_mel_ce: 3.2602744102478027 (3.2344424669372085) | > loss: 0.825185239315033 (0.8180851717618418) | > current_lr: 5e-06 | > step_time: 0.1744 (0.22942346259600696) | > loader_time: 0.0084 (0.009186985822075763) [1m --> TIME: 2024-12-19 14:41:20 -- STEP: 2598/4076 -- GLOBAL_STEP: 10750[0m | > loss_text_ce: 0.035446371883153915 (0.03790666505671444) | > loss_mel_ce: 3.1725833415985107 (3.234096481856981) | > loss: 0.8020074367523193 (0.818000786643105) | > current_lr: 5e-06 | > step_time: 0.2583 (0.22939010837428653) | > loader_time: 0.0091 (0.009179542026123332) [1m --> TIME: 2024-12-19 14:41:41 -- STEP: 2648/4076 -- GLOBAL_STEP: 10800[0m | > loss_text_ce: 0.03796885162591934 (0.03789253988137238) | > loss_mel_ce: 3.3604373931884766 (3.233906840539775) | > loss: 0.8496015667915344 (0.8179498450363507) | > current_lr: 5e-06 | > step_time: 0.1926 (0.22940110908174077) | > loader_time: 0.0082 (0.009175846313781972) [1m --> TIME: 2024-12-19 14:42:01 -- STEP: 2698/4076 -- GLOBAL_STEP: 10850[0m | > loss_text_ce: 0.0393529050052166 (0.037895539471723555) | > loss_mel_ce: 3.2670581340789795 (3.2340980559653794) | > loss: 0.8266027569770813 (0.8179983987995566) | > current_lr: 5e-06 | > step_time: 0.2119 (0.22933941597934998) | > loader_time: 0.0086 (0.009170999961751235) [1m --> TIME: 2024-12-19 14:42:22 -- STEP: 2748/4076 -- GLOBAL_STEP: 10900[0m | > loss_text_ce: 0.03616093844175339 (0.03789612061847027) | > loss_mel_ce: 3.3859670162200928 (3.2339098238355075) | > loss: 0.8555319905281067 (0.8179514860352051) | > current_lr: 5e-06 | > step_time: 0.1575 (0.22948406851447822) | > loader_time: 0.0088 (0.009170229257975093) [1m --> TIME: 2024-12-19 14:42:43 -- STEP: 2798/4076 -- GLOBAL_STEP: 10950[0m | > loss_text_ce: 0.03769762068986893 (0.037905123048835414) | > loss_mel_ce: 3.1699626445770264 (3.2334510844634927) | > loss: 0.8019150495529175 (0.817839051737284) | > current_lr: 5e-06 | > step_time: 0.278 (0.2294654016242528) | > loader_time: 0.0089 (0.009170008608918269) [1m --> TIME: 2024-12-19 14:43:04 -- STEP: 2848/4076 -- GLOBAL_STEP: 11000[0m | > loss_text_ce: 0.03878331929445267 (0.037900078191495164) | > loss_mel_ce: 3.160334825515747 (3.2328467467025463) | > loss: 0.7997795343399048 (0.8176867060698148) | > current_lr: 5e-06 | > step_time: 0.2693 (0.22944204358572368) | > loader_time: 0.0093 (0.009166887720649166) [1m --> TIME: 2024-12-19 14:43:25 -- STEP: 2898/4076 -- GLOBAL_STEP: 11050[0m | > loss_text_ce: 0.03934790566563606 (0.037895060346826284) | > loss_mel_ce: 3.1399269104003906 (3.2317984726118043) | > loss: 0.7948186993598938 (0.8174233830657966) | > current_lr: 5e-06 | > step_time: 0.2324 (0.22933704668772964) | > loader_time: 0.0099 (0.009161727275414) [1m --> TIME: 2024-12-19 14:43:45 -- STEP: 2948/4076 -- GLOBAL_STEP: 11100[0m | > loss_text_ce: 0.03815104812383652 (0.03788104864264482) | > loss_mel_ce: 3.075192451477051 (3.231574148303622) | > loss: 0.7783358693122864 (0.8173637991035645) | > current_lr: 5e-06 | > step_time: 0.1734 (0.22920785870610288) | > loader_time: 0.0088 (0.009160044801445472) [1m --> TIME: 2024-12-19 14:44:06 -- STEP: 2998/4076 -- GLOBAL_STEP: 11150[0m | > loss_text_ce: 0.03292595222592354 (0.0378666998839641) | > loss_mel_ce: 3.470226526260376 (3.231533929377579) | > loss: 0.8757880926132202 (0.8173501571867764) | > current_lr: 5e-06 | > step_time: 0.23 (0.2292927027225812) | > loader_time: 0.0086 (0.009153302468165625) [1m --> TIME: 2024-12-19 14:44:27 -- STEP: 3048/4076 -- GLOBAL_STEP: 11200[0m | > loss_text_ce: 0.04326223209500313 (0.03785973387951243) | > loss_mel_ce: 3.0603606700897217 (3.230165794687325) | > loss: 0.7759057283401489 (0.8170063819863349) | > current_lr: 5e-06 | > step_time: 0.193 (0.22912003088185165) | > loader_time: 0.009 (0.009145292553688913) [1m --> TIME: 2024-12-19 14:44:48 -- STEP: 3098/4076 -- GLOBAL_STEP: 11250[0m | > loss_text_ce: 0.03816128522157669 (0.03785192172401969) | > loss_mel_ce: 3.116966724395752 (3.2299450413498936) | > loss: 0.788782000541687 (0.8169492406180165) | > current_lr: 5e-06 | > step_time: 0.1892 (0.22929994439678544) | > loader_time: 0.0082 (0.009139316631487066) [1m --> TIME: 2024-12-19 14:45:09 -- STEP: 3148/4076 -- GLOBAL_STEP: 11300[0m | > loss_text_ce: 0.0343550480902195 (0.03783960432569191) | > loss_mel_ce: 3.240449905395508 (3.2305930635526225) | > loss: 0.8187012672424316 (0.8171081668691378) | > current_lr: 5e-06 | > step_time: 0.1547 (0.229244623887039) | > loader_time: 0.0079 (0.00913809677242778) [1m --> TIME: 2024-12-19 14:45:29 -- STEP: 3198/4076 -- GLOBAL_STEP: 11350[0m | > loss_text_ce: 0.03613537549972534 (0.0378258742125305) | > loss_mel_ce: 3.3643453121185303 (3.229720013748492) | > loss: 0.8501201868057251 (0.8168864718372182) | > current_lr: 5e-06 | > step_time: 0.2278 (0.2290843027393992) | > loader_time: 0.0084 (0.00912860664000281) [1m --> TIME: 2024-12-19 14:45:51 -- STEP: 3248/4076 -- GLOBAL_STEP: 11400[0m | > loss_text_ce: 0.03623725846409798 (0.037828239193071427) | > loss_mel_ce: 3.0272393226623535 (3.2295809832259383) | > loss: 0.765869140625 (0.8168523054698416) | > current_lr: 5e-06 | > step_time: 0.2646 (0.22921468200061118) | > loader_time: 0.0092 (0.009127765673721944) [1m --> TIME: 2024-12-19 14:46:12 -- STEP: 3298/4076 -- GLOBAL_STEP: 11450[0m | > loss_text_ce: 0.03852559253573418 (0.037819510045093666) | > loss_mel_ce: 3.0930426120758057 (3.2297645093745535) | > loss: 0.7828920483589172 (0.8168960047028436) | > current_lr: 5e-06 | > step_time: 0.3284 (0.22926283598379915) | > loader_time: 0.0098 (0.009133605612921957) [1m --> TIME: 2024-12-19 14:46:32 -- STEP: 3348/4076 -- GLOBAL_STEP: 11500[0m | > loss_text_ce: 0.04083014279603958 (0.03780606736706095) | > loss_mel_ce: 3.0790321826934814 (3.22953511836851) | > loss: 0.7799655795097351 (0.8168352963213708) | > current_lr: 5e-06 | > step_time: 0.1573 (0.22913909869951726) | > loader_time: 0.0084 (0.009132609073976177) [1m --> TIME: 2024-12-19 14:46:54 -- STEP: 3398/4076 -- GLOBAL_STEP: 11550[0m | > loss_text_ce: 0.04170304164290428 (0.037794466928555356) | > loss_mel_ce: 3.008549690246582 (3.229597424253991) | > loss: 0.762563169002533 (0.8168479727077931) | > current_lr: 5e-06 | > step_time: 0.2072 (0.22921343246861858) | > loader_time: 0.0088 (0.009137631655441026) [1m --> TIME: 2024-12-19 14:47:15 -- STEP: 3448/4076 -- GLOBAL_STEP: 11600[0m | > loss_text_ce: 0.0436948761343956 (0.037794651519269494) | > loss_mel_ce: 3.30785870552063 (3.229507851628309) | > loss: 0.8378884196281433 (0.8168256257222034) | > current_lr: 5e-06 | > step_time: 0.1897 (0.22921224661050427) | > loader_time: 0.0088 (0.009131083701436867) [1m --> TIME: 2024-12-19 14:47:36 -- STEP: 3498/4076 -- GLOBAL_STEP: 11650[0m | > loss_text_ce: 0.03717725723981857 (0.037795438391799226) | > loss_mel_ce: 3.274270534515381 (3.229176621494329) | > loss: 0.8278619647026062 (0.816743014965008) | > current_lr: 5e-06 | > step_time: 0.2327 (0.22926967732902248) | > loader_time: 0.0088 (0.009138160667942883) [1m --> TIME: 2024-12-19 14:47:56 -- STEP: 3548/4076 -- GLOBAL_STEP: 11700[0m | > loss_text_ce: 0.037933312356472015 (0.03778134773379317) | > loss_mel_ce: 3.42116641998291 (3.229040280874851) | > loss: 0.8647749423980713 (0.8167054071568849) | > current_lr: 5e-06 | > step_time: 0.1572 (0.2291998813493679) | > loader_time: 0.0088 (0.0091429407190712) [1m --> TIME: 2024-12-19 14:48:17 -- STEP: 3598/4076 -- GLOBAL_STEP: 11750[0m | > loss_text_ce: 0.04346802458167076 (0.03777455967364875) | > loss_mel_ce: 3.4087297916412354 (3.22881817718292) | > loss: 0.8630494475364685 (0.816648184192916) | > current_lr: 5e-06 | > step_time: 0.1856 (0.22915165990243158) | > loader_time: 0.0109 (0.009144777520091211) [1m --> TIME: 2024-12-19 14:48:37 -- STEP: 3648/4076 -- GLOBAL_STEP: 11800[0m | > loss_text_ce: 0.036960355937480927 (0.037776772187115934) | > loss_mel_ce: 3.0791821479797363 (3.228643566435369) | > loss: 0.7790356278419495 (0.8166050846083904) | > current_lr: 5e-06 | > step_time: 0.1755 (0.22904328067313157) | > loader_time: 0.0083 (0.009142701497726266) [1m --> TIME: 2024-12-19 14:48:58 -- STEP: 3698/4076 -- GLOBAL_STEP: 11850[0m | > loss_text_ce: 0.03822722285985947 (0.037770878878185865) | > loss_mel_ce: 3.2285923957824707 (3.2281894202229813) | > loss: 0.8167049288749695 (0.8164900746972188) | > current_lr: 5e-06 | > step_time: 0.2119 (0.22898270459739883) | > loader_time: 0.0104 (0.00914793146049866) [1m --> TIME: 2024-12-19 14:49:19 -- STEP: 3748/4076 -- GLOBAL_STEP: 11900[0m | > loss_text_ce: 0.038399580866098404 (0.03776293634084177) | > loss_mel_ce: 2.961857318878174 (3.2279484981915516) | > loss: 0.7500642538070679 (0.8164278586239033) | > current_lr: 5e-06 | > step_time: 0.1772 (0.22896849415727202) | > loader_time: 0.0087 (0.009151575086974275) [1m --> TIME: 2024-12-19 14:49:40 -- STEP: 3798/4076 -- GLOBAL_STEP: 11950[0m | > loss_text_ce: 0.03583686053752899 (0.03774853121624646) | > loss_mel_ce: 2.9572715759277344 (3.227594347738356) | > loss: 0.7482771277427673 (0.8163357197202586) | > current_lr: 5e-06 | > step_time: 0.2919 (0.22896377597123838) | > loader_time: 0.0087 (0.009155577455714471) [1m --> TIME: 2024-12-19 14:50:00 -- STEP: 3848/4076 -- GLOBAL_STEP: 12000[0m | > loss_text_ce: 0.03788788616657257 (0.037740791950320644) | > loss_mel_ce: 3.3192198276519775 (3.2276491361075803) | > loss: 0.839276909828186 (0.8163474819733296) | > current_lr: 5e-06 | > step_time: 0.2358 (0.22892083199752836) | > loader_time: 0.0106 (0.009156926877781153) [1m --> TIME: 2024-12-19 14:50:22 -- STEP: 3898/4076 -- GLOBAL_STEP: 12050[0m | > loss_text_ce: 0.03972601145505905 (0.03773308792015326) | > loss_mel_ce: 3.387324810028076 (3.227318991019583) | > loss: 0.856762707233429 (0.8162630196716185) | > current_lr: 5e-06 | > step_time: 0.2754 (0.2289896920866205) | > loader_time: 0.0087 (0.009153476001177762) [1m --> TIME: 2024-12-19 14:50:43 -- STEP: 3948/4076 -- GLOBAL_STEP: 12100[0m | > loss_text_ce: 0.037743087857961655 (0.03772935956875062) | > loss_mel_ce: 3.2392520904541016 (3.2266304449469874) | > loss: 0.8192487955093384 (0.8160899510529747) | > current_lr: 5e-06 | > step_time: 0.2452 (0.22899262043963567) | > loader_time: 0.009 (0.009153447373418212) [1m --> TIME: 2024-12-19 14:51:03 -- STEP: 3998/4076 -- GLOBAL_STEP: 12150[0m | > loss_text_ce: 0.03913554549217224 (0.037733503276777616) | > loss_mel_ce: 3.100931167602539 (3.2262736353771664) | > loss: 0.785016655921936 (0.816001784554) | > current_lr: 5e-06 | > step_time: 0.1877 (0.22891615294646833) | > loader_time: 0.0091 (0.009151542407384562) [1m --> TIME: 2024-12-19 14:51:24 -- STEP: 4048/4076 -- GLOBAL_STEP: 12200[0m | > loss_text_ce: 0.03353722393512726 (0.037725834324109295) | > loss_mel_ce: 3.0572757720947266 (3.2260808908774474) | > loss: 0.772703230381012 (0.815951681184203) | > current_lr: 5e-06 | > step_time: 0.1749 (0.22900240211618744) | > loader_time: 0.01 (0.009154779637755135) [1m > EVALUATION [0m [1m--> EVAL PERFORMANCE[0m | > avg_loader_time:[92m 0.46798714307638317 [0m(-0.0026920850460345758) | > avg_loss_text_ce:[92m 0.03606826195923182 [0m(-0.0007612109184265067) | > avg_loss_mel_ce:[92m 3.2282389310690074 [0m(-0.03236612906822778) | > avg_loss:[92m 3.264307196323688 [0m(-0.03312732623173664) > BEST MODEL : /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb/best_model_12228.pth [4m[1m > EPOCH: 3/8[0m --> /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb [1m > TRAINING (2024-12-19 14:52:50) [0m [1m --> TIME: 2024-12-19 14:53:01 -- STEP: 22/4076 -- GLOBAL_STEP: 12250[0m | > loss_text_ce: 0.03839646652340889 (0.037084355103698646) | > loss_mel_ce: 3.5568878650665283 (3.2088912291960283) | > loss: 0.8988210558891296 (0.8114938925613057) | > current_lr: 5e-06 | > step_time: 0.182 (0.2259328690442172) | > loader_time: 0.0103 (0.009886969219554554) [1m --> TIME: 2024-12-19 14:53:21 -- STEP: 72/4076 -- GLOBAL_STEP: 12300[0m | > loss_text_ce: 0.04052743315696716 (0.03748296703108483) | > loss_mel_ce: 3.411214828491211 (3.2127027677165136) | > loss: 0.8629355430603027 (0.8125464328461223) | > current_lr: 5e-06 | > step_time: 0.1932 (0.22528858648406136) | > loader_time: 0.0105 (0.009594377544191148) [1m --> TIME: 2024-12-19 14:53:43 -- STEP: 122/4076 -- GLOBAL_STEP: 12350[0m | > loss_text_ce: 0.03364793211221695 (0.037508113981514705) | > loss_mel_ce: 3.2150604724884033 (3.209727115318423) | > loss: 0.8121771216392517 (0.8118088064623661) | > current_lr: 5e-06 | > step_time: 0.2298 (0.2284822307649206) | > loader_time: 0.0081 (0.009366850383946157) [1m --> TIME: 2024-12-19 14:54:03 -- STEP: 172/4076 -- GLOBAL_STEP: 12400[0m | > loss_text_ce: 0.039146602153778076 (0.037361952909376744) | > loss_mel_ce: 3.346857786178589 (3.2015712344369223) | > loss: 0.8465011119842529 (0.8097332958565202) | > current_lr: 5e-06 | > step_time: 0.1923 (0.2271481891011083) | > loader_time: 0.0093 (0.009153898372206584) [1m --> TIME: 2024-12-19 14:54:24 -- STEP: 222/4076 -- GLOBAL_STEP: 12450[0m | > loss_text_ce: 0.03493984043598175 (0.03731476339335379) | > loss_mel_ce: 3.544938564300537 (3.2029412512306696) | > loss: 0.8949695825576782 (0.8100640032742474) | > current_lr: 5e-06 | > step_time: 0.2577 (0.2286654313405355) | > loader_time: 0.0083 (0.009130993404903925) [1m --> TIME: 2024-12-19 14:54:45 -- STEP: 272/4076 -- GLOBAL_STEP: 12500[0m | > loss_text_ce: 0.03797873482108116 (0.03727422334829018) | > loss_mel_ce: 3.0085201263427734 (3.204876959323883) | > loss: 0.7616246938705444 (0.8105377949335996) | > current_lr: 5e-06 | > step_time: 0.1539 (0.22820201779113097) | > loader_time: 0.0084 (0.009093033916809977) [1m --> TIME: 2024-12-19 14:55:07 -- STEP: 322/4076 -- GLOBAL_STEP: 12550[0m | > loss_text_ce: 0.03446922451257706 (0.03727906317050969) | > loss_mel_ce: 3.180114984512329 (3.204239623146768) | > loss: 0.8036460280418396 (0.8103796710138738) | > current_lr: 5e-06 | > step_time: 0.2921 (0.22915631791819696) | > loader_time: 0.0084 (0.009022015962541473) [1m --> TIME: 2024-12-19 14:55:27 -- STEP: 372/4076 -- GLOBAL_STEP: 12600[0m | > loss_text_ce: 0.0367378406226635 (0.03727173658528478) | > loss_mel_ce: 3.634937286376953 (3.20758089006588) | > loss: 0.9179188013076782 (0.8112131557477419) | > current_lr: 5e-06 | > step_time: 0.1908 (0.22798158020101567) | > loader_time: 0.0086 (0.009019563915908977) [1m --> TIME: 2024-12-19 14:55:48 -- STEP: 422/4076 -- GLOBAL_STEP: 12650[0m | > loss_text_ce: 0.03532775864005089 (0.037327379999037916) | > loss_mel_ce: 3.4518680572509766 (3.2089665953016957) | > loss: 0.8717989325523376 (0.8115734931134501) | > current_lr: 5e-06 | > step_time: 0.2334 (0.22773714596626318) | > loader_time: 0.0085 (0.009014983877751508) [1m --> TIME: 2024-12-19 14:56:09 -- STEP: 472/4076 -- GLOBAL_STEP: 12700[0m | > loss_text_ce: 0.036434486508369446 (0.03737830918013926) | > loss_mel_ce: 3.202056884765625 (3.201881861787731) | > loss: 0.8096228241920471 (0.8098150420997103) | > current_lr: 5e-06 | > step_time: 0.1921 (0.22870582746247115) | > loader_time: 0.008 (0.00896018591977783) [1m --> TIME: 2024-12-19 14:56:30 -- STEP: 522/4076 -- GLOBAL_STEP: 12750[0m | > loss_text_ce: 0.03943972662091255 (0.037312810676764736) | > loss_mel_ce: 3.739812135696411 (3.2044068086192983) | > loss: 0.9448129534721375 (0.810429904538553) | > current_lr: 5e-06 | > step_time: 0.1882 (0.22882403952865307) | > loader_time: 0.0102 (0.008971584711038297) [1m --> TIME: 2024-12-19 14:56:51 -- STEP: 572/4076 -- GLOBAL_STEP: 12800[0m | > loss_text_ce: 0.03726429492235184 (0.03730626899905349) | > loss_mel_ce: 3.75293231010437 (3.200018300043119) | > loss: 0.9475491642951965 (0.8093311422979916) | > current_lr: 5e-06 | > step_time: 0.1926 (0.22832166356640263) | > loader_time: 0.0083 (0.008967582579259274) [1m --> TIME: 2024-12-19 14:57:11 -- STEP: 622/4076 -- GLOBAL_STEP: 12850[0m | > loss_text_ce: 0.03905683383345604 (0.03733044800459382) | > loss_mel_ce: 3.1900835037231445 (3.1999846551195974) | > loss: 0.8072850704193115 (0.8093287759846814) | > current_lr: 5e-06 | > step_time: 0.2136 (0.2282065632258964) | > loader_time: 0.0088 (0.008973615345847567) [1m --> TIME: 2024-12-19 14:57:32 -- STEP: 672/4076 -- GLOBAL_STEP: 12900[0m | > loss_text_ce: 0.03481433168053627 (0.037319895126746015) | > loss_mel_ce: 3.0991313457489014 (3.1990603652028797) | > loss: 0.7834864258766174 (0.8090950654198727) | > current_lr: 5e-06 | > step_time: 0.1752 (0.22823622822761536) | > loader_time: 0.0081 (0.008977591281845457) [1m --> TIME: 2024-12-19 14:57:53 -- STEP: 722/4076 -- GLOBAL_STEP: 12950[0m | > loss_text_ce: 0.03336639702320099 (0.03736390651995707) | > loss_mel_ce: 3.2122929096221924 (3.200814330346695) | > loss: 0.8114148378372192 (0.8095445594133764) | > current_lr: 5e-06 | > step_time: 0.1832 (0.22806566871104147) | > loader_time: 0.0078 (0.008994322734526317) [1m --> TIME: 2024-12-19 14:58:14 -- STEP: 772/4076 -- GLOBAL_STEP: 13000[0m | > loss_text_ce: 0.0351848341524601 (0.037331982281715324) | > loss_mel_ce: 3.3862521648406982 (3.196311486817393) | > loss: 0.8553592562675476 (0.8084108673537953) | > current_lr: 5e-06 | > step_time: 0.1948 (0.22871453712641265) | > loader_time: 0.0098 (0.009003448980459895) [1m --> TIME: 2024-12-19 14:58:35 -- STEP: 822/4076 -- GLOBAL_STEP: 13050[0m | > loss_text_ce: 0.03989504650235176 (0.03732138285481368) | > loss_mel_ce: 3.0316498279571533 (3.197035519166003) | > loss: 0.767886221408844 (0.8085892253403537) | > current_lr: 5e-06 | > step_time: 0.2271 (0.22809180552071898) | > loader_time: 0.0085 (0.008991488575065227) [1m --> TIME: 2024-12-19 14:58:56 -- STEP: 872/4076 -- GLOBAL_STEP: 13100[0m | > loss_text_ce: 0.038484249264001846 (0.03733829488559123) | > loss_mel_ce: 3.107943534851074 (3.1951632751237353) | > loss: 0.7866069674491882 (0.8081253925594716) | > current_lr: 5e-06 | > step_time: 0.1903 (0.228242584324758) | > loader_time: 0.008 (0.008970462127563065) [1m --> TIME: 2024-12-19 14:59:16 -- STEP: 922/4076 -- GLOBAL_STEP: 13150[0m | > loss_text_ce: 0.03532971441745758 (0.03732923372222391) | > loss_mel_ce: 3.3922760486602783 (3.1969414106418657) | > loss: 0.8569014668464661 (0.808567661202135) | > current_lr: 5e-06 | > step_time: 0.2305 (0.22814833083535482) | > loader_time: 0.0097 (0.008972330362315817) [1m --> TIME: 2024-12-19 14:59:37 -- STEP: 972/4076 -- GLOBAL_STEP: 13200[0m | > loss_text_ce: 0.03328896313905716 (0.03730099557787794) | > loss_mel_ce: 3.3772342205047607 (3.1973434004763996) | > loss: 0.8526307940483093 (0.8086610992013673) | > current_lr: 5e-06 | > step_time: 0.1732 (0.22825142189308448) | > loader_time: 0.0089 (0.00898327282917353) [1m --> TIME: 2024-12-19 14:59:58 -- STEP: 1022/4076 -- GLOBAL_STEP: 13250[0m | > loss_text_ce: 0.035376403480768204 (0.037276718204152105) | > loss_mel_ce: 3.3393118381500244 (3.2008572301286065) | > loss: 0.8436720371246338 (0.8095334872107685) | > current_lr: 5e-06 | > step_time: 0.2131 (0.2281292886883312) | > loader_time: 0.0101 (0.008992532694643032) [1m --> TIME: 2024-12-19 15:00:19 -- STEP: 1072/4076 -- GLOBAL_STEP: 13300[0m | > loss_text_ce: 0.0371418334543705 (0.037262278362246826) | > loss_mel_ce: 3.240931510925293 (3.201349601149558) | > loss: 0.8195183277130127 (0.8096529698527571) | > current_lr: 5e-06 | > step_time: 0.1728 (0.22819954690648547) | > loader_time: 0.0088 (0.008983092743959007) [1m --> TIME: 2024-12-19 15:00:40 -- STEP: 1122/4076 -- GLOBAL_STEP: 13350[0m | > loss_text_ce: 0.03756885603070259 (0.037278351282664386) | > loss_mel_ce: 3.4418158531188965 (3.199173031006266) | > loss: 0.8698461651802063 (0.809112845470136) | > current_lr: 5e-06 | > step_time: 0.2781 (0.2283244868225805) | > loader_time: 0.0085 (0.008987524292685775) [1m --> TIME: 2024-12-19 15:01:01 -- STEP: 1172/4076 -- GLOBAL_STEP: 13400[0m | > loss_text_ce: 0.03737704083323479 (0.03729867874787735) | > loss_mel_ce: 3.0434157848358154 (3.199276905824705) | > loss: 0.7701982259750366 (0.8091438959889848) | > current_lr: 5e-06 | > step_time: 0.1755 (0.2284691203576306) | > loader_time: 0.0081 (0.008996220782348322) [1m --> TIME: 2024-12-19 15:01:22 -- STEP: 1222/4076 -- GLOBAL_STEP: 13450[0m | > loss_text_ce: 0.0348762571811676 (0.037282357156983956) | > loss_mel_ce: 3.221870183944702 (3.1972963556330227) | > loss: 0.8141866326332092 (0.8086446782668559) | > current_lr: 5e-06 | > step_time: 0.2132 (0.22851098851954682) | > loader_time: 0.0086 (0.008995740971666702) [1m --> TIME: 2024-12-19 15:01:43 -- STEP: 1272/4076 -- GLOBAL_STEP: 13500[0m | > loss_text_ce: 0.03586922213435173 (0.03727868621665162) | > loss_mel_ce: 3.1785941123962402 (3.197684459903703) | > loss: 0.8036158084869385 (0.8087407865520536) | > current_lr: 5e-06 | > step_time: 0.2635 (0.2282616282409092) | > loader_time: 0.0098 (0.009006566596481039) [1m --> TIME: 2024-12-19 15:02:04 -- STEP: 1322/4076 -- GLOBAL_STEP: 13550[0m | > loss_text_ce: 0.0389779731631279 (0.037270374365049486) | > loss_mel_ce: 3.380887031555176 (3.196492194768698) | > loss: 0.8549662232398987 (0.808440642349658) | > current_lr: 5e-06 | > step_time: 0.2907 (0.22840870508808592) | > loader_time: 0.0101 (0.009006916304399315) [1m --> TIME: 2024-12-19 15:02:24 -- STEP: 1372/4076 -- GLOBAL_STEP: 13600[0m | > loss_text_ce: 0.03891277313232422 (0.03727613169287195) | > loss_mel_ce: 3.0862817764282227 (3.1942526783261953) | > loss: 0.7812986373901367 (0.8078822025230946) | > current_lr: 5e-06 | > step_time: 0.2367 (0.22828346230198274) | > loader_time: 0.0084 (0.0090077971230443) [1m --> TIME: 2024-12-19 15:02:46 -- STEP: 1422/4076 -- GLOBAL_STEP: 13650[0m | > loss_text_ce: 0.039964161813259125 (0.03727037528511424) | > loss_mel_ce: 3.1328577995300293 (3.193974598885921) | > loss: 0.7932054996490479 (0.807811243540794) | > current_lr: 5e-06 | > step_time: 0.2317 (0.22847799963085963) | > loader_time: 0.0088 (0.009006671932343868) [1m --> TIME: 2024-12-19 15:03:07 -- STEP: 1472/4076 -- GLOBAL_STEP: 13700[0m | > loss_text_ce: 0.0399545356631279 (0.03727376209976879) | > loss_mel_ce: 3.183540105819702 (3.194856956925079) | > loss: 0.8058736324310303 (0.808032679776458) | > current_lr: 5e-06 | > step_time: 0.2344 (0.22882730627189513) | > loader_time: 0.0083 (0.009009429132160936) [1m --> TIME: 2024-12-19 15:03:28 -- STEP: 1522/4076 -- GLOBAL_STEP: 13750[0m | > loss_text_ce: 0.03785393387079239 (0.037251180828440054) | > loss_mel_ce: 2.9064671993255615 (3.195710899638751) | > loss: 0.7360802888870239 (0.8082405201865552) | > current_lr: 5e-06 | > step_time: 0.187 (0.22864415814779435) | > loader_time: 0.0096 (0.00900920712524895) [1m --> TIME: 2024-12-19 15:03:49 -- STEP: 1572/4076 -- GLOBAL_STEP: 13800[0m | > loss_text_ce: 0.03007730096578598 (0.03724527436263539) | > loss_mel_ce: 3.1731772422790527 (3.1953960590386794) | > loss: 0.800813615322113 (0.808160333346774) | > current_lr: 5e-06 | > step_time: 0.1737 (0.2286678648176994) | > loader_time: 0.0083 (0.00900474516793365) [1m --> TIME: 2024-12-19 15:04:10 -- STEP: 1622/4076 -- GLOBAL_STEP: 13850[0m | > loss_text_ce: 0.03903914615511894 (0.03722612895117443) | > loss_mel_ce: 3.1835999488830566 (3.1935990031638632) | > loss: 0.8056597709655762 (0.807706282957561) | > current_lr: 5e-06 | > step_time: 0.2139 (0.22883766955717866) | > loader_time: 0.0085 (0.009008267069862458) [1m --> TIME: 2024-12-19 15:04:31 -- STEP: 1672/4076 -- GLOBAL_STEP: 13900[0m | > loss_text_ce: 0.042071953415870667 (0.037213340214493774) | > loss_mel_ce: 3.282872200012207 (3.193439937807154) | > loss: 0.8312360644340515 (0.8076633194630791) | > current_lr: 5e-06 | > step_time: 0.2179 (0.2290299958304355) | > loader_time: 0.0084 (0.009002244501022636) [1m --> TIME: 2024-12-19 15:04:52 -- STEP: 1722/4076 -- GLOBAL_STEP: 13950[0m | > loss_text_ce: 0.03966793790459633 (0.03722542572796834) | > loss_mel_ce: 3.1610453128814697 (3.1929451349028013) | > loss: 0.8001782894134521 (0.8075426402269197) | > current_lr: 5e-06 | > step_time: 0.2337 (0.22903420271635333) | > loader_time: 0.0089 (0.0089996245956864) [1m --> TIME: 2024-12-19 15:05:13 -- STEP: 1772/4076 -- GLOBAL_STEP: 14000[0m | > loss_text_ce: 0.033675454556941986 (0.037209558140898574) | > loss_mel_ce: 3.140345335006714 (3.1937408390755557) | > loss: 0.7935051918029785 (0.807737599309895) | > current_lr: 5e-06 | > step_time: 0.1569 (0.22893449347928738) | > loader_time: 0.0084 (0.009002129730198788) [1m --> TIME: 2024-12-19 15:05:33 -- STEP: 1822/4076 -- GLOBAL_STEP: 14050[0m | > loss_text_ce: 0.03496367856860161 (0.03722268063268341) | > loss_mel_ce: 3.374213457107544 (3.1935015530800825) | > loss: 0.8522942662239075 (0.8076810583259607) | > current_lr: 5e-06 | > step_time: 0.2912 (0.22869706939002163) | > loader_time: 0.0084 (0.008992376887575867) [1m --> TIME: 2024-12-19 15:05:54 -- STEP: 1872/4076 -- GLOBAL_STEP: 14100[0m | > loss_text_ce: 0.04239707812666893 (0.037220691830222336) | > loss_mel_ce: 2.9193432331085205 (3.1920091919919344) | > loss: 0.7404350638389587 (0.8073074709082766) | > current_lr: 5e-06 | > step_time: 0.15 (0.2288190587463542) | > loader_time: 0.0094 (0.008996098087384162) [1m --> TIME: 2024-12-19 15:06:15 -- STEP: 1922/4076 -- GLOBAL_STEP: 14150[0m | > loss_text_ce: 0.038922522217035294 (0.03721953106852219) | > loss_mel_ce: 3.2446045875549316 (3.19090788719185) | > loss: 0.8208817839622498 (0.8070318545684849) | > current_lr: 5e-06 | > step_time: 0.2095 (0.2286533073877817) | > loader_time: 0.0088 (0.008982330281578178) [1m --> TIME: 2024-12-19 15:06:36 -- STEP: 1972/4076 -- GLOBAL_STEP: 14200[0m | > loss_text_ce: 0.037560753524303436 (0.037213831416446505) | > loss_mel_ce: 3.1524384021759033 (3.190170292317503) | > loss: 0.7974997758865356 (0.8068460310085788) | > current_lr: 5e-06 | > step_time: 0.1768 (0.22885081540740285) | > loader_time: 0.0101 (0.008990041745361896) [1m --> TIME: 2024-12-19 15:06:57 -- STEP: 2022/4076 -- GLOBAL_STEP: 14250[0m | > loss_text_ce: 0.03524351865053177 (0.03720329159043519) | > loss_mel_ce: 3.218125104904175 (3.189123322418958) | > loss: 0.8133421540260315 (0.8065816536619445) | > current_lr: 5e-06 | > step_time: 0.2875 (0.22879179365910832) | > loader_time: 0.0094 (0.008994856291071962) [1m --> TIME: 2024-12-19 15:07:17 -- STEP: 2072/4076 -- GLOBAL_STEP: 14300[0m | > loss_text_ce: 0.03747757524251938 (0.03720102966025935) | > loss_mel_ce: 2.83514142036438 (3.1884016716802424) | > loss: 0.7181547284126282 (0.8064006754848023) | > current_lr: 5e-06 | > step_time: 0.1578 (0.22850672492189297) | > loader_time: 0.0082 (0.008989894712293481) [1m --> TIME: 2024-12-19 15:07:37 -- STEP: 2122/4076 -- GLOBAL_STEP: 14350[0m | > loss_text_ce: 0.037843070924282074 (0.03720165153528224) | > loss_mel_ce: 3.0519189834594727 (3.1879572089722856) | > loss: 0.772440493106842 (0.8062897152658003) | > current_lr: 5e-06 | > step_time: 0.2334 (0.2284706380432445) | > loader_time: 0.012 (0.008982102338375635) [1m --> TIME: 2024-12-19 15:07:58 -- STEP: 2172/4076 -- GLOBAL_STEP: 14400[0m | > loss_text_ce: 0.04038911312818527 (0.03720544395680177) | > loss_mel_ce: 2.8635599613189697 (3.1871876396087844) | > loss: 0.7259872555732727 (0.8060982710858509) | > current_lr: 5e-06 | > step_time: 0.1735 (0.2284705490697155) | > loader_time: 0.0092 (0.008985462987840077) [1m --> TIME: 2024-12-19 15:08:20 -- STEP: 2222/4076 -- GLOBAL_STEP: 14450[0m | > loss_text_ce: 0.040016088634729385 (0.037204869948226985) | > loss_mel_ce: 3.005319833755493 (3.1871969708443517) | > loss: 0.7613340020179749 (0.8061004603746014) | > current_lr: 5e-06 | > step_time: 0.2323 (0.22854123192794892) | > loader_time: 0.0087 (0.008995079650844583) [1m --> TIME: 2024-12-19 15:08:41 -- STEP: 2272/4076 -- GLOBAL_STEP: 14500[0m | > loss_text_ce: 0.034903258085250854 (0.037199916269689236) | > loss_mel_ce: 3.387376546859741 (3.187023523617795) | > loss: 0.8555699586868286 (0.8060558601741633) | > current_lr: 5e-06 | > step_time: 0.1773 (0.22866128259141674) | > loader_time: 0.0089 (0.008996914075294024) [1m --> TIME: 2024-12-19 15:09:03 -- STEP: 2322/4076 -- GLOBAL_STEP: 14550[0m | > loss_text_ce: 0.03351062536239624 (0.037178209321019916) | > loss_mel_ce: 3.1554768085479736 (3.1863574528878944) | > loss: 0.7972468733787537 (0.8058839157467552) | > current_lr: 5e-06 | > step_time: 0.3596 (0.22903108299035224) | > loader_time: 0.0101 (0.009004387949993661) [1m --> TIME: 2024-12-19 15:09:24 -- STEP: 2372/4076 -- GLOBAL_STEP: 14600[0m | > loss_text_ce: 0.038705579936504364 (0.03718113438124199) | > loss_mel_ce: 2.569655179977417 (3.1859864597577823) | > loss: 0.6520901918411255 (0.8057918987326428) | > current_lr: 5e-06 | > step_time: 0.1918 (0.22904238461644005) | > loader_time: 0.0086 (0.009004287936836237) [1m --> TIME: 2024-12-19 15:09:45 -- STEP: 2422/4076 -- GLOBAL_STEP: 14650[0m | > loss_text_ce: 0.03693990781903267 (0.037187670369061114) | > loss_mel_ce: 3.3055689334869385 (3.186515582483917) | > loss: 0.8356271982192993 (0.8059258133689785) | > current_lr: 5e-06 | > step_time: 0.2769 (0.22910496530997462) | > loader_time: 0.0092 (0.009000192290982569) [1m --> TIME: 2024-12-19 15:10:06 -- STEP: 2472/4076 -- GLOBAL_STEP: 14700[0m | > loss_text_ce: 0.04061702638864517 (0.037186684495314315) | > loss_mel_ce: 3.2748918533325195 (3.1862176732916594) | > loss: 0.8288772106170654 (0.8058510896023415) | > current_lr: 5e-06 | > step_time: 0.1503 (0.2290738057164312) | > loader_time: 0.0084 (0.008998882712669742) [1m --> TIME: 2024-12-19 15:10:27 -- STEP: 2522/4076 -- GLOBAL_STEP: 14750[0m | > loss_text_ce: 0.03595678508281708 (0.03716807498712561) | > loss_mel_ce: 3.0851497650146484 (3.1863186861389057) | > loss: 0.7802766561508179 (0.8058716904587627) | > current_lr: 5e-06 | > step_time: 0.2281 (0.22899470397350238) | > loader_time: 0.0082 (0.009000452617914495) [1m --> TIME: 2024-12-19 15:10:48 -- STEP: 2572/4076 -- GLOBAL_STEP: 14800[0m | > loss_text_ce: 0.0346447117626667 (0.037162138526450984) | > loss_mel_ce: 2.941718339920044 (3.186666676916558) | > loss: 0.744090735912323 (0.8059572040756593) | > current_lr: 5e-06 | > step_time: 0.152 (0.22917546528865163) | > loader_time: 0.0082 (0.009003419620252874) [1m --> TIME: 2024-12-19 15:11:09 -- STEP: 2622/4076 -- GLOBAL_STEP: 14850[0m | > loss_text_ce: 0.03725957125425339 (0.03715657041082049) | > loss_mel_ce: 3.1181371212005615 (3.1864633286481165) | > loss: 0.7888491749763489 (0.8059049749492586) | > current_lr: 5e-06 | > step_time: 0.2336 (0.22919934901638303) | > loader_time: 0.0088 (0.00900326431661412) [1m --> TIME: 2024-12-19 15:11:30 -- STEP: 2672/4076 -- GLOBAL_STEP: 14900[0m | > loss_text_ce: 0.03546992316842079 (0.037158588717780786) | > loss_mel_ce: 3.1822426319122314 (3.1865227501727804) | > loss: 0.8044281601905823 (0.8059203348786163) | > current_lr: 5e-06 | > step_time: 0.1788 (0.22916643687351018) | > loader_time: 0.0086 (0.009009339584561876) [1m --> TIME: 2024-12-19 15:11:51 -- STEP: 2722/4076 -- GLOBAL_STEP: 14950[0m | > loss_text_ce: 0.03880176320672035 (0.03715312089031994) | > loss_mel_ce: 3.2748546600341797 (3.185801725979672) | > loss: 0.8284140825271606 (0.8057387118870319) | > current_lr: 5e-06 | > step_time: 0.3193 (0.22913720360755216) | > loader_time: 0.0104 (0.009010485127776266) [1m --> TIME: 2024-12-19 15:12:12 -- STEP: 2772/4076 -- GLOBAL_STEP: 15000[0m | > loss_text_ce: 0.03871038183569908 (0.03714564211954176) | > loss_mel_ce: 3.2159335613250732 (3.1860692666206467) | > loss: 0.8136609792709351 (0.8058037272994469) | > current_lr: 5e-06 | > step_time: 0.2915 (0.22907517964361718) | > loader_time: 0.0094 (0.00901494148317697) [1m --> TIME: 2024-12-19 15:12:33 -- STEP: 2822/4076 -- GLOBAL_STEP: 15050[0m | > loss_text_ce: 0.03398142382502556 (0.03713769742705501) | > loss_mel_ce: 3.2895724773406982 (3.1853436899894803) | > loss: 0.8308884501457214 (0.8056203469919184) | > current_lr: 5e-06 | > step_time: 0.2902 (0.22908353492944986) | > loader_time: 0.0094 (0.009009254392709052) [1m --> TIME: 2024-12-19 15:12:53 -- STEP: 2872/4076 -- GLOBAL_STEP: 15100[0m | > loss_text_ce: 0.033943772315979004 (0.037129424459853504) | > loss_mel_ce: 3.208828926086426 (3.1860159203036527) | > loss: 0.8106931447982788 (0.8057863362610828) | > current_lr: 5e-06 | > step_time: 0.1572 (0.22908075456831778) | > loader_time: 0.0084 (0.00901073350215688) [1m --> TIME: 2024-12-19 15:13:14 -- STEP: 2922/4076 -- GLOBAL_STEP: 15150[0m | > loss_text_ce: 0.03322417289018631 (0.03711651922455398) | > loss_mel_ce: 3.134290933609009 (3.185150785449425) | > loss: 0.7918787598609924 (0.8055668262526392) | > current_lr: 5e-06 | > step_time: 0.2778 (0.22896870475040565) | > loader_time: 0.0106 (0.009010861704889673) [1m --> TIME: 2024-12-19 15:13:35 -- STEP: 2972/4076 -- GLOBAL_STEP: 15200[0m | > loss_text_ce: 0.03640322387218475 (0.037110916486769605) | > loss_mel_ce: 3.042912006378174 (3.1845869023257545) | > loss: 0.7698287963867188 (0.8054244547159792) | > current_lr: 5e-06 | > step_time: 0.1756 (0.22901174815153527) | > loader_time: 0.0082 (0.009012167899797939) [1m --> TIME: 2024-12-19 15:13:55 -- STEP: 3022/4076 -- GLOBAL_STEP: 15250[0m | > loss_text_ce: 0.03639349341392517 (0.03711472069912408) | > loss_mel_ce: 3.047131299972534 (3.1842966415801586) | > loss: 0.770881175994873 (0.8053528406145072) | > current_lr: 5e-06 | > step_time: 0.2091 (0.2289571720270981) | > loader_time: 0.008 (0.009008218246449866) [1m --> TIME: 2024-12-19 15:14:16 -- STEP: 3072/4076 -- GLOBAL_STEP: 15300[0m | > loss_text_ce: 0.037001896649599075 (0.03710531664122153) | > loss_mel_ce: 3.0778684616088867 (3.1841954169794864) | > loss: 0.778717577457428 (0.8053251833771345) | > current_lr: 5e-06 | > step_time: 0.1908 (0.22891127139640347) | > loader_time: 0.009 (0.009008873098840322) [1m --> TIME: 2024-12-19 15:14:37 -- STEP: 3122/4076 -- GLOBAL_STEP: 15350[0m | > loss_text_ce: 0.03581149876117706 (0.03709581876525626) | > loss_mel_ce: 2.815040349960327 (3.18395049605592) | > loss: 0.7127129435539246 (0.8052615786614429) | > current_lr: 5e-06 | > step_time: 0.2146 (0.22887045881978824) | > loader_time: 0.0093 (0.009006533998468609) [1m --> TIME: 2024-12-19 15:14:58 -- STEP: 3172/4076 -- GLOBAL_STEP: 15400[0m | > loss_text_ce: 0.03979892283678055 (0.037102514530284264) | > loss_mel_ce: 2.8666141033172607 (3.1836102117054947) | > loss: 0.7266032695770264 (0.8051781815019853) | > current_lr: 5e-06 | > step_time: 0.2077 (0.2288965104056186) | > loader_time: 0.0098 (0.009015487918035981) [1m --> TIME: 2024-12-19 15:15:19 -- STEP: 3222/4076 -- GLOBAL_STEP: 15450[0m | > loss_text_ce: 0.036788538098335266 (0.037094221101214714) | > loss_mel_ce: 3.285029649734497 (3.183531079656358) | > loss: 0.8304545283317566 (0.8051563251062912) | > current_lr: 5e-06 | > step_time: 0.2322 (0.22895660909490506) | > loader_time: 0.0088 (0.009013945015967077) [1m --> TIME: 2024-12-19 15:15:41 -- STEP: 3272/4076 -- GLOBAL_STEP: 15500[0m | > loss_text_ce: 0.040604475885629654 (0.03709142778266883) | > loss_mel_ce: 3.577021360397339 (3.1831571555953686) | > loss: 0.9044064879417419 (0.805062145759831) | > current_lr: 5e-06 | > step_time: 0.1939 (0.22910703678585786) | > loader_time: 0.0088 (0.009009420361670418) [1m --> TIME: 2024-12-19 15:16:02 -- STEP: 3322/4076 -- GLOBAL_STEP: 15550[0m | > loss_text_ce: 0.036604467779397964 (0.03708529327410139) | > loss_mel_ce: 3.5055601596832275 (3.1825867442751656) | > loss: 0.8855411410331726 (0.8049180093369942) | > current_lr: 5e-06 | > step_time: 0.3178 (0.229139227381964) | > loader_time: 0.0102 (0.009010291759828565) [1m --> TIME: 2024-12-19 15:16:24 -- STEP: 3372/4076 -- GLOBAL_STEP: 15600[0m | > loss_text_ce: 0.03714217245578766 (0.037092839366030486) | > loss_mel_ce: 3.0045957565307617 (3.182734468441412) | > loss: 0.7604345083236694 (0.8049568268398647) | > current_lr: 5e-06 | > step_time: 0.1904 (0.229382952125047) | > loader_time: 0.0088 (0.009015040748342773) [1m --> TIME: 2024-12-19 15:16:45 -- STEP: 3422/4076 -- GLOBAL_STEP: 15650[0m | > loss_text_ce: 0.037872232496738434 (0.037088173151469055) | > loss_mel_ce: 3.2710013389587402 (3.1824039081465307) | > loss: 0.8272184133529663 (0.8048730202232577) | > current_lr: 5e-06 | > step_time: 0.2755 (0.2294227539761452) | > loader_time: 0.0103 (0.009013336907347201) [1m --> TIME: 2024-12-19 15:17:06 -- STEP: 3472/4076 -- GLOBAL_STEP: 15700[0m | > loss_text_ce: 0.04086204990744591 (0.03707718933945353) | > loss_mel_ce: 3.553478956222534 (3.182330529398628) | > loss: 0.8985852599143982 (0.8048519296442849) | > current_lr: 5e-06 | > step_time: 0.2188 (0.22944509056031975) | > loader_time: 0.0092 (0.009013706981311742) [1m --> TIME: 2024-12-19 15:17:28 -- STEP: 3522/4076 -- GLOBAL_STEP: 15750[0m | > loss_text_ce: 0.03376162052154541 (0.03706887455880298) | > loss_mel_ce: 3.059166431427002 (3.1819387833412724) | > loss: 0.7732319831848145 (0.8047519144789856) | > current_lr: 5e-06 | > step_time: 0.1807 (0.22961231350289493) | > loader_time: 0.0091 (0.009016241288063269) [1m --> TIME: 2024-12-19 15:17:49 -- STEP: 3572/4076 -- GLOBAL_STEP: 15800[0m | > loss_text_ce: 0.032582107931375504 (0.03706380457086494) | > loss_mel_ce: 3.563277244567871 (3.1818250908296246) | > loss: 0.8989648222923279 (0.8047222238271785) | > current_lr: 5e-06 | > step_time: 0.1792 (0.22964508616243468) | > loader_time: 0.0088 (0.0090118622592956) [1m --> TIME: 2024-12-19 15:18:11 -- STEP: 3622/4076 -- GLOBAL_STEP: 15850[0m | > loss_text_ce: 0.040881745517253876 (0.03705716274024565) | > loss_mel_ce: 3.103771448135376 (3.180911846545303) | > loss: 0.7861632704734802 (0.8044922522817894) | > current_lr: 5e-06 | > step_time: 0.2792 (0.2298010910893592) | > loader_time: 0.0097 (0.009010995178180534) [1m --> TIME: 2024-12-19 15:18:32 -- STEP: 3672/4076 -- GLOBAL_STEP: 15900[0m | > loss_text_ce: 0.040332794189453125 (0.03705364242114917) | > loss_mel_ce: 3.4570538997650146 (3.1805200290446143) | > loss: 0.8743466734886169 (0.8043934178410783) | > current_lr: 5e-06 | > step_time: 0.1927 (0.22988151237855547) | > loader_time: 0.0085 (0.009008906960227657) [1m --> TIME: 2024-12-19 15:18:53 -- STEP: 3722/4076 -- GLOBAL_STEP: 15950[0m | > loss_text_ce: 0.03401626646518707 (0.03705947963878195) | > loss_mel_ce: 3.07214617729187 (3.1806254943553465) | > loss: 0.7765406370162964 (0.8044212434895244) | > current_lr: 5e-06 | > step_time: 0.2295 (0.2299129711819361) | > loader_time: 0.0084 (0.00900378181113152) [1m --> TIME: 2024-12-19 15:19:14 -- STEP: 3772/4076 -- GLOBAL_STEP: 16000[0m | > loss_text_ce: 0.03694145753979683 (0.03705551033525516) | > loss_mel_ce: 3.0419139862060547 (3.18070771161976) | > loss: 0.7697138786315918 (0.8044408054771496) | > current_lr: 5e-06 | > step_time: 0.2421 (0.22998238305145455) | > loader_time: 0.0086 (0.009001131528002845) [1m --> TIME: 2024-12-19 15:19:36 -- STEP: 3822/4076 -- GLOBAL_STEP: 16050[0m | > loss_text_ce: 0.03450879082083702 (0.03705703609826234) | > loss_mel_ce: 2.83489727973938 (3.180123809680206) | > loss: 0.717351496219635 (0.8042952114377944) | > current_lr: 5e-06 | > step_time: 0.187 (0.23001011156524304) | > loader_time: 0.01 (0.00899860716689513) [1m --> TIME: 2024-12-19 15:19:56 -- STEP: 3872/4076 -- GLOBAL_STEP: 16100[0m | > loss_text_ce: 0.03610030189156532 (0.0370544978149989) | > loss_mel_ce: 3.1506030559539795 (3.179392714084176) | > loss: 0.7966758608818054 (0.8041118029658944) | > current_lr: 5e-06 | > step_time: 0.2358 (0.22991737366215254) | > loader_time: 0.0083 (0.009001557923052928) [1m --> TIME: 2024-12-19 15:20:18 -- STEP: 3922/4076 -- GLOBAL_STEP: 16150[0m | > loss_text_ce: 0.0367722325026989 (0.03704987694297982) | > loss_mel_ce: 3.2622554302215576 (3.1790040870639227) | > loss: 0.824756920337677 (0.8040134909656318) | > current_lr: 5e-06 | > step_time: 0.2862 (0.23002068465124398) | > loader_time: 0.01 (0.009002423967286562) [1m --> TIME: 2024-12-19 15:20:38 -- STEP: 3972/4076 -- GLOBAL_STEP: 16200[0m | > loss_text_ce: 0.03592563048005104 (0.03704665358434356) | > loss_mel_ce: 3.4752438068389893 (3.1794377329486334) | > loss: 0.8777923583984375 (0.8041210965987771) | > current_lr: 5e-06 | > step_time: 0.2346 (0.22998267952409032) | > loader_time: 0.0088 (0.009002772402787382) [1m --> TIME: 2024-12-19 15:21:00 -- STEP: 4022/4076 -- GLOBAL_STEP: 16250[0m | > loss_text_ce: 0.03575780615210533 (0.037033800067206815) | > loss_mel_ce: 2.924914598464966 (3.17922276824579) | > loss: 0.7401680946350098 (0.8040641420430527) | > current_lr: 5e-06 | > step_time: 0.2113 (0.23005996083573044) | > loader_time: 0.0082 (0.009000875948674053) [1m --> TIME: 2024-12-19 15:21:21 -- STEP: 4072/4076 -- GLOBAL_STEP: 16300[0m | > loss_text_ce: 0.03514046221971512 (0.037030912206594165) | > loss_mel_ce: 3.4160873889923096 (3.1786754838024427) | > loss: 0.8628069758415222 (0.8039265989320443) | > current_lr: 5e-06 | > step_time: 0.1768 (0.23009231791983417) | > loader_time: 0.0078 (0.009002135287099368) [1m > EVALUATION [0m [1m--> EVAL PERFORMANCE[0m | > avg_loader_time:[92m 0.46561657465421236 [0m(-0.0023705684221708156) | > avg_loss_text_ce:[92m 0.03552525476194345 [0m(-0.0005430071972883713) | > avg_loss_mel_ce:[92m 3.2022439333108754 [0m(-0.025994997758131966) | > avg_loss:[92m 3.2377691910817075 [0m(-0.026538005241980578) > BEST MODEL : /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb/best_model_16304.pth [4m[1m > EPOCH: 4/8[0m --> /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb [1m > TRAINING (2024-12-19 15:22:35) [0m [1m --> TIME: 2024-12-19 15:22:56 -- STEP: 46/4076 -- GLOBAL_STEP: 16350[0m | > loss_text_ce: 0.03475036472082138 (0.036996674480969494) | > loss_mel_ce: 3.225905656814575 (3.179445603619451) | > loss: 0.8151640295982361 (0.8041105672069218) | > current_lr: 5e-06 | > step_time: 0.2323 (0.2308712212935738) | > loader_time: 0.0087 (0.010067929392275599) [1m --> TIME: 2024-12-19 15:23:17 -- STEP: 96/4076 -- GLOBAL_STEP: 16400[0m | > loss_text_ce: 0.040823910385370255 (0.03694573962517702) | > loss_mel_ce: 3.0758984088897705 (3.185656023522218) | > loss: 0.7791805863380432 (0.8056504415969054) | > current_lr: 5e-06 | > step_time: 0.1541 (0.23046576976776123) | > loader_time: 0.0081 (0.009650178253650664) [1m --> TIME: 2024-12-19 15:23:38 -- STEP: 146/4076 -- GLOBAL_STEP: 16450[0m | > loss_text_ce: 0.03896167129278183 (0.03691387617935057) | > loss_mel_ce: 3.1762940883636475 (3.179566520534149) | > loss: 0.8038139343261719 (0.8041200992995745) | > current_lr: 5e-06 | > step_time: 0.227 (0.2306879121963292) | > loader_time: 0.0088 (0.009575139986325617) [1m --> TIME: 2024-12-19 15:24:00 -- STEP: 196/4076 -- GLOBAL_STEP: 16500[0m | > loss_text_ce: 0.0353238582611084 (0.036938024342668276) | > loss_mel_ce: 3.3078622817993164 (3.1769372699212055) | > loss: 0.8357965350151062 (0.8034688234329224) | > current_lr: 5e-06 | > step_time: 0.1748 (0.23398923387332837) | > loader_time: 0.0085 (0.009539287917468013) [1m --> TIME: 2024-12-19 15:24:20 -- STEP: 246/4076 -- GLOBAL_STEP: 16550[0m | > loss_text_ce: 0.04211759939789772 (0.03693334424368491) | > loss_mel_ce: 3.4417073726654053 (3.1759488020485978) | > loss: 0.8709562420845032 (0.8032205364083856) | > current_lr: 5e-06 | > step_time: 0.2653 (0.23296147342619858) | > loader_time: 0.0085 (0.009449557560246164) [1m --> TIME: 2024-12-19 15:24:42 -- STEP: 296/4076 -- GLOBAL_STEP: 16600[0m | > loss_text_ce: 0.0387364886701107 (0.036819870797665535) | > loss_mel_ce: 3.061490297317505 (3.1767450997958315) | > loss: 0.7750567197799683 (0.803391242550837) | > current_lr: 5e-06 | > step_time: 0.2165 (0.23243369927277435) | > loader_time: 0.009 (0.009370759532258317) [1m --> TIME: 2024-12-19 15:25:03 -- STEP: 346/4076 -- GLOBAL_STEP: 16650[0m | > loss_text_ce: 0.03283879533410072 (0.03685074966573129) | > loss_mel_ce: 3.2318053245544434 (3.172930958643125) | > loss: 0.816161036491394 (0.8024454277030307) | > current_lr: 5e-06 | > step_time: 0.2146 (0.23263409992173917) | > loader_time: 0.0099 (0.009356947992578413) [1m --> TIME: 2024-12-19 15:25:24 -- STEP: 396/4076 -- GLOBAL_STEP: 16700[0m | > loss_text_ce: 0.040516581386327744 (0.03679355438516446) | > loss_mel_ce: 2.9795753955841064 (3.1722531794297573) | > loss: 0.7550230026245117 (0.8022616841275284) | > current_lr: 5e-06 | > step_time: 0.1931 (0.23183621541418212) | > loader_time: 0.0097 (0.009337864138863308) [1m --> TIME: 2024-12-19 15:25:45 -- STEP: 446/4076 -- GLOBAL_STEP: 16750[0m | > loss_text_ce: 0.03969000652432442 (0.03679417423285844) | > loss_mel_ce: 3.0775146484375 (3.173521893975981) | > loss: 0.7793011665344238 (0.802579017765319) | > current_lr: 5e-06 | > step_time: 0.2337 (0.23204207901462845) | > loader_time: 0.01 (0.009357549684464669) [1m --> TIME: 2024-12-19 15:26:06 -- STEP: 496/4076 -- GLOBAL_STEP: 16800[0m | > loss_text_ce: 0.03861211612820625 (0.036743973501987985) | > loss_mel_ce: 2.9148929119110107 (3.1721274319194976) | > loss: 0.738376259803772 (0.8022178522040768) | > current_lr: 5e-06 | > step_time: 0.1746 (0.23207276771145482) | > loader_time: 0.0107 (0.009386122226715091) [1m --> TIME: 2024-12-19 15:26:28 -- STEP: 546/4076 -- GLOBAL_STEP: 16850[0m | > loss_text_ce: 0.03463263064622879 (0.03669895353512122) | > loss_mel_ce: 2.9564852714538574 (3.1696024954100666) | > loss: 0.7477794885635376 (0.8015753632281727) | > current_lr: 5e-06 | > step_time: 0.209 (0.23254214843987547) | > loader_time: 0.01 (0.009385287980020267) [1m --> TIME: 2024-12-19 15:26:49 -- STEP: 596/4076 -- GLOBAL_STEP: 16900[0m | > loss_text_ce: 0.038248784840106964 (0.03666568815945678) | > loss_mel_ce: 3.102968692779541 (3.1668608768674344) | > loss: 0.7853043675422668 (0.8008816421231969) | > current_lr: 5e-06 | > step_time: 0.1521 (0.23290529067084292) | > loader_time: 0.0095 (0.009381196642882072) [1m --> TIME: 2024-12-19 15:27:10 -- STEP: 646/4076 -- GLOBAL_STEP: 16950[0m | > loss_text_ce: 0.036956787109375 (0.03666698182111276) | > loss_mel_ce: 3.0137765407562256 (3.167550801123626) | > loss: 0.7626833319664001 (0.8010544466898537) | > current_lr: 5e-06 | > step_time: 0.2123 (0.23297480569904433) | > loader_time: 0.009 (0.009380412913697416) [1m --> TIME: 2024-12-19 15:27:31 -- STEP: 696/4076 -- GLOBAL_STEP: 17000[0m | > loss_text_ce: 0.029752880334854126 (0.036693029723868786) | > loss_mel_ce: 2.9037694931030273 (3.1681455760851653) | > loss: 0.7333806157112122 (0.801209652475242) | > current_lr: 5e-06 | > step_time: 0.1572 (0.23247544107765988) | > loader_time: 0.0078 (0.009361964532698705) [1m --> TIME: 2024-12-19 15:27:52 -- STEP: 746/4076 -- GLOBAL_STEP: 17050[0m | > loss_text_ce: 0.041343655437231064 (0.03667554395378195) | > loss_mel_ce: 3.3009932041168213 (3.1648266139682413) | > loss: 0.8355842232704163 (0.8003755405984679) | > current_lr: 5e-06 | > step_time: 0.2615 (0.23224130616430944) | > loader_time: 0.0092 (0.009339790881159467) [1m --> TIME: 2024-12-19 15:28:13 -- STEP: 796/4076 -- GLOBAL_STEP: 17100[0m | > loss_text_ce: 0.03859998658299446 (0.03668947030448509) | > loss_mel_ce: 3.0238940715789795 (3.1615675523053444) | > loss: 0.7656235098838806 (0.799564256440455) | > current_lr: 5e-06 | > step_time: 0.174 (0.23235772602522192) | > loader_time: 0.0095 (0.009330892083632863) [1m --> TIME: 2024-12-19 15:28:34 -- STEP: 846/4076 -- GLOBAL_STEP: 17150[0m | > loss_text_ce: 0.03669162839651108 (0.03665300018107848) | > loss_mel_ce: 3.169971227645874 (3.161833333912751) | > loss: 0.801665723323822 (0.7996215842153446) | > current_lr: 5e-06 | > step_time: 0.2576 (0.23195307023699965) | > loader_time: 0.0087 (0.009310874814964617) [1m --> TIME: 2024-12-19 15:28:55 -- STEP: 896/4076 -- GLOBAL_STEP: 17200[0m | > loss_text_ce: 0.04039214551448822 (0.03664591998261) | > loss_mel_ce: 3.643089771270752 (3.164105423859186) | > loss: 0.9208704829216003 (0.8001878367337797) | > current_lr: 5e-06 | > step_time: 0.156 (0.2318004005189453) | > loader_time: 0.0092 (0.009305986176644037) [1m --> TIME: 2024-12-19 15:29:16 -- STEP: 946/4076 -- GLOBAL_STEP: 17250[0m | > loss_text_ce: 0.0334460623562336 (0.03663151293410181) | > loss_mel_ce: 3.1540896892547607 (3.1635424380070547) | > loss: 0.7968839406967163 (0.8000434885317377) | > current_lr: 5e-06 | > step_time: 0.2287 (0.2316921231106522) | > loader_time: 0.0102 (0.009314275443175868) [1m --> TIME: 2024-12-19 15:29:37 -- STEP: 996/4076 -- GLOBAL_STEP: 17300[0m | > loss_text_ce: 0.03291749581694603 (0.03664618318182337) | > loss_mel_ce: 3.1805145740509033 (3.1636880806172214) | > loss: 0.8033580183982849 (0.8000835666216042) | > current_lr: 5e-06 | > step_time: 0.1746 (0.2314046401096635) | > loader_time: 0.0099 (0.009300354733524538) [1m --> TIME: 2024-12-19 15:29:57 -- STEP: 1046/4076 -- GLOBAL_STEP: 17350[0m | > loss_text_ce: 0.03593864291906357 (0.036633266914229884) | > loss_mel_ce: 3.2487237453460693 (3.1649558377767377) | > loss: 0.8211656212806702 (0.8003972767643901) | > current_lr: 5e-06 | > step_time: 0.2094 (0.23116987180071624) | > loader_time: 0.0081 (0.009289247811866297) [1m --> TIME: 2024-12-19 15:30:18 -- STEP: 1096/4076 -- GLOBAL_STEP: 17400[0m | > loss_text_ce: 0.037493400275707245 (0.03663076046364803) | > loss_mel_ce: 2.6355583667755127 (3.1641180556621005) | > loss: 0.6682629585266113 (0.8001872047129339) | > current_lr: 5e-06 | > step_time: 0.1538 (0.23082581140699174) | > loader_time: 0.0085 (0.009285917899904452) [1m --> TIME: 2024-12-19 15:30:39 -- STEP: 1146/4076 -- GLOBAL_STEP: 17450[0m | > loss_text_ce: 0.03689340129494667 (0.036650237191735026) | > loss_mel_ce: 3.1055359840393066 (3.1617711680513825) | > loss: 0.7856073379516602 (0.7996053520921639) | > current_lr: 5e-06 | > step_time: 0.2776 (0.23058488814202482) | > loader_time: 0.0103 (0.009286433198273065) [1m --> TIME: 2024-12-19 15:31:00 -- STEP: 1196/4076 -- GLOBAL_STEP: 17500[0m | > loss_text_ce: 0.03861837834119797 (0.03663562079517785) | > loss_mel_ce: 3.3332605361938477 (3.16333892672357) | > loss: 0.8429697155952454 (0.7999936376128314) | > current_lr: 5e-06 | > step_time: 0.2332 (0.23072874147357744) | > loader_time: 0.0097 (0.009285890737106155) [1m --> TIME: 2024-12-19 15:31:21 -- STEP: 1246/4076 -- GLOBAL_STEP: 17550[0m | > loss_text_ce: 0.04206618294119835 (0.03665628686044991) | > loss_mel_ce: 3.1028318405151367 (3.1617937258311684) | > loss: 0.7862244844436646 (0.7996125038515893) | > current_lr: 5e-06 | > step_time: 0.1834 (0.23049019982688496) | > loader_time: 0.0089 (0.009271635863792435) [1m --> TIME: 2024-12-19 15:31:42 -- STEP: 1296/4076 -- GLOBAL_STEP: 17600[0m | > loss_text_ce: 0.03574691340327263 (0.03665847406883573) | > loss_mel_ce: 2.9778850078582764 (3.161055741302764) | > loss: 0.7534079551696777 (0.7994285545507333) | > current_lr: 5e-06 | > step_time: 0.1576 (0.2305088611664595) | > loader_time: 0.0083 (0.009259209220792026) [1m --> TIME: 2024-12-19 15:32:03 -- STEP: 1346/4076 -- GLOBAL_STEP: 17650[0m | > loss_text_ce: 0.03777245432138443 (0.0366519505385016) | > loss_mel_ce: 3.233812093734741 (3.1598633545468884) | > loss: 0.8178961277008057 (0.7991288269788338) | > current_lr: 5e-06 | > step_time: 0.2089 (0.23058664887302163) | > loader_time: 0.0099 (0.009252324678214114) [1m --> TIME: 2024-12-19 15:32:24 -- STEP: 1396/4076 -- GLOBAL_STEP: 17700[0m | > loss_text_ce: 0.03398666903376579 (0.03664201354063556) | > loss_mel_ce: 3.1519484519958496 (3.159865098047396) | > loss: 0.7964837551116943 (0.7991267785131079) | > current_lr: 5e-06 | > step_time: 0.1914 (0.23053497571316692) | > loader_time: 0.0091 (0.009243471574646688) [1m --> TIME: 2024-12-19 15:32:45 -- STEP: 1446/4076 -- GLOBAL_STEP: 17750[0m | > loss_text_ce: 0.036155298352241516 (0.036624781045428934) | > loss_mel_ce: 3.1992101669311523 (3.15919801242454) | > loss: 0.808841347694397 (0.7989556989729159) | > current_lr: 5e-06 | > step_time: 0.2123 (0.23081163713711092) | > loader_time: 0.0101 (0.009246249746319985) [1m --> TIME: 2024-12-19 15:33:06 -- STEP: 1496/4076 -- GLOBAL_STEP: 17800[0m | > loss_text_ce: 0.03892911598086357 (0.03662177976494763) | > loss_mel_ce: 2.981112241744995 (3.157799862444722) | > loss: 0.7550103664398193 (0.7986054111292019) | > current_lr: 5e-06 | > step_time: 0.1563 (0.2305726534861294) | > loader_time: 0.0095 (0.009253420772399486) [1m --> TIME: 2024-12-19 15:33:27 -- STEP: 1546/4076 -- GLOBAL_STEP: 17850[0m | > loss_text_ce: 0.03911208361387253 (0.03661881959400731) | > loss_mel_ce: 3.2491934299468994 (3.1570865113244815) | > loss: 0.8220763802528381 (0.7984263332895599) | > current_lr: 5e-06 | > step_time: 0.2111 (0.230567850856855) | > loader_time: 0.0088 (0.009252812853143246) [1m --> TIME: 2024-12-19 15:33:47 -- STEP: 1596/4076 -- GLOBAL_STEP: 17900[0m | > loss_text_ce: 0.03616901859641075 (0.03662919090771323) | > loss_mel_ce: 3.1948931217193604 (3.155740516974516) | > loss: 0.807765543460846 (0.7980924275958151) | > current_lr: 5e-06 | > step_time: 0.1565 (0.2302787517545217) | > loader_time: 0.0086 (0.009230794763206533) [1m --> TIME: 2024-12-19 15:34:09 -- STEP: 1646/4076 -- GLOBAL_STEP: 17950[0m | > loss_text_ce: 0.03704625740647316 (0.03663930253217385) | > loss_mel_ce: 3.251009225845337 (3.1557133270208073) | > loss: 0.8220138549804688 (0.7980881578505113) | > current_lr: 5e-06 | > step_time: 0.2884 (0.23037678126431319) | > loader_time: 0.0086 (0.009210388251961648) [1m --> TIME: 2024-12-19 15:34:30 -- STEP: 1696/4076 -- GLOBAL_STEP: 18000[0m | > loss_text_ce: 0.038298413157463074 (0.03665031896649316) | > loss_mel_ce: 3.024139642715454 (3.155064975596826) | > loss: 0.7656095027923584 (0.7979288241070398) | > current_lr: 5e-06 | > step_time: 0.191 (0.2304767730101099) | > loader_time: 0.0089 (0.009207843047267975) [1m --> TIME: 2024-12-19 15:34:50 -- STEP: 1746/4076 -- GLOBAL_STEP: 18050[0m | > loss_text_ce: 0.03497316315770149 (0.03665472166602079) | > loss_mel_ce: 3.267673969268799 (3.155409214969082) | > loss: 0.8256617784500122 (0.7980159846095013) | > current_lr: 5e-06 | > step_time: 0.1874 (0.23031485954647346) | > loader_time: 0.0103 (0.009205770656415261) [1m --> TIME: 2024-12-19 15:35:12 -- STEP: 1796/4076 -- GLOBAL_STEP: 18100[0m | > loss_text_ce: 0.036327991634607315 (0.03665532475806161) | > loss_mel_ce: 2.941512107849121 (3.1548475818272976) | > loss: 0.7444600462913513 (0.7978757271389655) | > current_lr: 5e-06 | > step_time: 0.1927 (0.23056065053876096) | > loader_time: 0.0086 (0.009206231559039739) [1m --> TIME: 2024-12-19 15:35:33 -- STEP: 1846/4076 -- GLOBAL_STEP: 18150[0m | > loss_text_ce: 0.03655578941106796 (0.0366530238084681) | > loss_mel_ce: 3.144113063812256 (3.153760056454339) | > loss: 0.7951672077178955 (0.7976032705374169) | > current_lr: 5e-06 | > step_time: 0.2328 (0.23057755333889082) | > loader_time: 0.0109 (0.009211407322537458) [1m --> TIME: 2024-12-19 15:35:54 -- STEP: 1896/4076 -- GLOBAL_STEP: 18200[0m | > loss_text_ce: 0.03854617848992348 (0.036636628003256566) | > loss_mel_ce: 3.3648178577423096 (3.1542585635487055) | > loss: 0.8508409857749939 (0.7977237983406346) | > current_lr: 5e-06 | > step_time: 0.192 (0.2305828639726598) | > loader_time: 0.0101 (0.00922429611914267) [1m --> TIME: 2024-12-19 15:36:15 -- STEP: 1946/4076 -- GLOBAL_STEP: 18250[0m | > loss_text_ce: 0.03498147055506706 (0.036622946395635445) | > loss_mel_ce: 3.287132978439331 (3.1541205315036147) | > loss: 0.8305286169052124 (0.7976858699186979) | > current_lr: 5e-06 | > step_time: 0.2929 (0.23058206547446017) | > loader_time: 0.0103 (0.009227279033837299) [1m --> TIME: 2024-12-19 15:36:36 -- STEP: 1996/4076 -- GLOBAL_STEP: 18300[0m | > loss_text_ce: 0.035822004079818726 (0.0365993479212638) | > loss_mel_ce: 3.3161914348602295 (3.1537233890894676) | > loss: 0.8380033373832703 (0.7975806846527873) | > current_lr: 5e-06 | > step_time: 0.156 (0.23032918745625708) | > loader_time: 0.0099 (0.009226251938538928) [1m --> TIME: 2024-12-19 15:36:57 -- STEP: 2046/4076 -- GLOBAL_STEP: 18350[0m | > loss_text_ce: 0.03665022924542427 (0.03659257271717635) | > loss_mel_ce: 3.1632347106933594 (3.152978556363582) | > loss: 0.7999712228775024 (0.7973927825194068) | > current_lr: 5e-06 | > step_time: 0.2134 (0.23018026037300085) | > loader_time: 0.0101 (0.009232949185115021) [1m --> TIME: 2024-12-19 15:37:18 -- STEP: 2096/4076 -- GLOBAL_STEP: 18400[0m | > loss_text_ce: 0.037285033613443375 (0.03659719816864051) | > loss_mel_ce: 3.1986465454101562 (3.15194688669143) | > loss: 0.8089829087257385 (0.7971360213656462) | > current_lr: 5e-06 | > step_time: 0.2174 (0.2301275451447217) | > loader_time: 0.0106 (0.009239513341707115) [1m --> TIME: 2024-12-19 15:37:38 -- STEP: 2146/4076 -- GLOBAL_STEP: 18450[0m | > loss_text_ce: 0.03664199635386467 (0.03659599803853348) | > loss_mel_ce: 2.9868524074554443 (3.150623485182117) | > loss: 0.7558736205101013 (0.7968048709692072) | > current_lr: 5e-06 | > step_time: 0.2469 (0.23006241485990322) | > loader_time: 0.0091 (0.009240587404256416) [1m --> TIME: 2024-12-19 15:37:59 -- STEP: 2196/4076 -- GLOBAL_STEP: 18500[0m | > loss_text_ce: 0.03966623172163963 (0.03660621965738296) | > loss_mel_ce: 3.1760201454162598 (3.149193114921696) | > loss: 0.8039215803146362 (0.7964498337989296) | > current_lr: 5e-06 | > step_time: 0.1933 (0.2300904017545704) | > loader_time: 0.0099 (0.009243801968996624) [1m --> TIME: 2024-12-19 15:38:21 -- STEP: 2246/4076 -- GLOBAL_STEP: 18550[0m | > loss_text_ce: 0.04031003266572952 (0.0366130636487478) | > loss_mel_ce: 3.0062592029571533 (3.14997869883917) | > loss: 0.761642336845398 (0.7966479407764395) | > current_lr: 5e-06 | > step_time: 0.2108 (0.23021396567326843) | > loader_time: 0.0085 (0.009234886347769825) [1m --> TIME: 2024-12-19 15:38:42 -- STEP: 2296/4076 -- GLOBAL_STEP: 18600[0m | > loss_text_ce: 0.03735021501779556 (0.036618776371189835) | > loss_mel_ce: 3.0117757320404053 (3.1504124461150766) | > loss: 0.7622814774513245 (0.7967578058199186) | > current_lr: 5e-06 | > step_time: 0.234 (0.23021795265765982) | > loader_time: 0.0091 (0.009232917002269182) [1m --> TIME: 2024-12-19 15:39:03 -- STEP: 2346/4076 -- GLOBAL_STEP: 18650[0m | > loss_text_ce: 0.032594937831163406 (0.0366250729372523) | > loss_mel_ce: 3.1627371311187744 (3.14968023865221) | > loss: 0.7988330125808716 (0.7965763280611211) | > current_lr: 5e-06 | > step_time: 0.2616 (0.23018633059121976) | > loader_time: 0.0105 (0.009230618891508661) [1m --> TIME: 2024-12-19 15:39:24 -- STEP: 2396/4076 -- GLOBAL_STEP: 18700[0m | > loss_text_ce: 0.037663690745830536 (0.036620165940486074) | > loss_mel_ce: 3.2708122730255127 (3.1478195581491892) | > loss: 0.827118992805481 (0.7961099311485518) | > current_lr: 5e-06 | > step_time: 0.1542 (0.23010761232328344) | > loader_time: 0.0102 (0.009225796379509656) [1m --> TIME: 2024-12-19 15:39:44 -- STEP: 2446/4076 -- GLOBAL_STEP: 18750[0m | > loss_text_ce: 0.03956333175301552 (0.03662908065284334) | > loss_mel_ce: 3.2583444118499756 (3.148026993307832) | > loss: 0.824476957321167 (0.7961640185931625) | > current_lr: 5e-06 | > step_time: 0.2142 (0.2300064101729936) | > loader_time: 0.0089 (0.009230425780850732) [1m --> TIME: 2024-12-19 15:40:06 -- STEP: 2496/4076 -- GLOBAL_STEP: 18800[0m | > loss_text_ce: 0.03611265495419502 (0.03664015673581533) | > loss_mel_ce: 3.3407886028289795 (3.1483978848808873) | > loss: 0.844225287437439 (0.7962595104741367) | > current_lr: 5e-06 | > step_time: 0.1741 (0.23012643832808896) | > loader_time: 0.0106 (0.009229755745484255) [1m --> TIME: 2024-12-19 15:40:26 -- STEP: 2546/4076 -- GLOBAL_STEP: 18850[0m | > loss_text_ce: 0.034732040017843246 (0.0366376650893515) | > loss_mel_ce: 3.116562843322754 (3.1482249660372084) | > loss: 0.7878237366676331 (0.7962156577973694) | > current_lr: 5e-06 | > step_time: 0.209 (0.23003843087243556) | > loader_time: 0.0101 (0.009232041036061919) [1m --> TIME: 2024-12-19 15:40:47 -- STEP: 2596/4076 -- GLOBAL_STEP: 18900[0m | > loss_text_ce: 0.033764537423849106 (0.03662697145663695) | > loss_mel_ce: 2.9693784713745117 (3.1472212459163047) | > loss: 0.750785768032074 (0.7959620544833287) | > current_lr: 5e-06 | > step_time: 0.1576 (0.23005282226806437) | > loader_time: 0.0102 (0.009235101414755392) [1m --> TIME: 2024-12-19 15:41:08 -- STEP: 2646/4076 -- GLOBAL_STEP: 18950[0m | > loss_text_ce: 0.03609403595328331 (0.03662598150756241) | > loss_mel_ce: 3.0389392375946045 (3.1460237950875016) | > loss: 0.7687582969665527 (0.795662444284804) | > current_lr: 5e-06 | > step_time: 0.2274 (0.229997070468202) | > loader_time: 0.0082 (0.00923242365331784) [1m --> TIME: 2024-12-19 15:41:29 -- STEP: 2696/4076 -- GLOBAL_STEP: 19000[0m | > loss_text_ce: 0.03436717763543129 (0.03661459774671547) | > loss_mel_ce: 2.8484408855438232 (3.1461414458666783) | > loss: 0.7207019925117493 (0.7956890110092278) | > current_lr: 5e-06 | > step_time: 0.2116 (0.22998293252656304) | > loader_time: 0.0107 (0.00923429159806812) [1m --> TIME: 2024-12-19 15:41:50 -- STEP: 2746/4076 -- GLOBAL_STEP: 19050[0m | > loss_text_ce: 0.030448036268353462 (0.03660397259932272) | > loss_mel_ce: 3.146735668182373 (3.1462816583441717) | > loss: 0.7942959070205688 (0.7957214077979393) | > current_lr: 5e-06 | > step_time: 0.1876 (0.23007789712184906) | > loader_time: 0.0099 (0.009241734381062381) [1m --> TIME: 2024-12-19 15:42:11 -- STEP: 2796/4076 -- GLOBAL_STEP: 19100[0m | > loss_text_ce: 0.038812022656202316 (0.03660176396276664) | > loss_mel_ce: 3.332289695739746 (3.145194496037452) | > loss: 0.8427754044532776 (0.7954490651004817) | > current_lr: 5e-06 | > step_time: 0.1892 (0.22996672531736434) | > loader_time: 0.0084 (0.009238319420848606) [1m --> TIME: 2024-12-19 15:42:32 -- STEP: 2846/4076 -- GLOBAL_STEP: 19150[0m | > loss_text_ce: 0.03459213301539421 (0.036600769952971006) | > loss_mel_ce: 3.164635181427002 (3.1446318535091002) | > loss: 0.7998068332672119 (0.7953081558853159) | > current_lr: 5e-06 | > step_time: 0.2119 (0.23003303937650624) | > loader_time: 0.0089 (0.009233119197907104) [1m --> TIME: 2024-12-19 15:42:53 -- STEP: 2896/4076 -- GLOBAL_STEP: 19200[0m | > loss_text_ce: 0.03892599418759346 (0.03660213830044683) | > loss_mel_ce: 3.12400484085083 (3.1447308568170715) | > loss: 0.7907326817512512 (0.7953332488461096) | > current_lr: 5e-06 | > step_time: 0.2464 (0.23009348393145196) | > loader_time: 0.0096 (0.009228923291132583) [1m --> TIME: 2024-12-19 15:43:15 -- STEP: 2946/4076 -- GLOBAL_STEP: 19250[0m | > loss_text_ce: 0.03821279853582382 (0.03660826242446028) | > loss_mel_ce: 3.4134628772735596 (3.145780465510615) | > loss: 0.8629189133644104 (0.7955971820493662) | > current_lr: 5e-06 | > step_time: 0.2147 (0.23015482330192785) | > loader_time: 0.0104 (0.00922747428774914) [1m --> TIME: 2024-12-19 15:43:36 -- STEP: 2996/4076 -- GLOBAL_STEP: 19300[0m | > loss_text_ce: 0.03658086806535721 (0.03662395858963099) | > loss_mel_ce: 3.2079057693481445 (3.1454208956382304) | > loss: 0.8111216425895691 (0.7955112135975321) | > current_lr: 5e-06 | > step_time: 0.179 (0.23011026411094723) | > loader_time: 0.0105 (0.009229853570222539) [1m --> TIME: 2024-12-19 15:43:57 -- STEP: 3046/4076 -- GLOBAL_STEP: 19350[0m | > loss_text_ce: 0.0371653288602829 (0.0366196886367866) | > loss_mel_ce: 3.2142438888549805 (3.1467416573729237) | > loss: 0.8128523230552673 (0.7958403365320857) | > current_lr: 5e-06 | > step_time: 0.2068 (0.23028491432543116) | > loader_time: 0.0086 (0.009226772946428487) [1m --> TIME: 2024-12-19 15:44:18 -- STEP: 3096/4076 -- GLOBAL_STEP: 19400[0m | > loss_text_ce: 0.03391473367810249 (0.036617201547805) | > loss_mel_ce: 3.1192257404327393 (3.146115567410023) | > loss: 0.7882851362228394 (0.7956831922287783) | > current_lr: 5e-06 | > step_time: 0.2161 (0.2301041048010499) | > loader_time: 0.0084 (0.009223075646146632) [1m --> TIME: 2024-12-19 15:44:38 -- STEP: 3146/4076 -- GLOBAL_STEP: 19450[0m | > loss_text_ce: 0.03487715497612953 (0.036616226353310175) | > loss_mel_ce: 3.196254014968872 (3.1456416977186987) | > loss: 0.807782769203186 (0.7955644809358529) | > current_lr: 5e-06 | > step_time: 0.2085 (0.23005484602516507) | > loader_time: 0.0085 (0.009217594129146474) [1m --> TIME: 2024-12-19 15:44:59 -- STEP: 3196/4076 -- GLOBAL_STEP: 19500[0m | > loss_text_ce: 0.03686569258570671 (0.03660812242439155) | > loss_mel_ce: 2.971202850341797 (3.145232258735222) | > loss: 0.7520171403884888 (0.7954600951884955) | > current_lr: 5e-06 | > step_time: 0.2107 (0.23010774130218475) | > loader_time: 0.0089 (0.00920889337310505) [1m --> TIME: 2024-12-19 15:45:20 -- STEP: 3246/4076 -- GLOBAL_STEP: 19550[0m | > loss_text_ce: 0.03589075431227684 (0.036614971367374884) | > loss_mel_ce: 3.0545380115509033 (3.1450606130628183) | > loss: 0.7726072072982788 (0.7954188960794308) | > current_lr: 5e-06 | > step_time: 0.3287 (0.2301549550859707) | > loader_time: 0.0092 (0.009205933786510918) [1m --> TIME: 2024-12-19 15:45:41 -- STEP: 3296/4076 -- GLOBAL_STEP: 19600[0m | > loss_text_ce: 0.031056789681315422 (0.03660557969999999) | > loss_mel_ce: 3.0898783206939697 (3.1441810188941584) | > loss: 0.7802338004112244 (0.795196649609405) | > current_lr: 5e-06 | > step_time: 0.1738 (0.23013610566414683) | > loader_time: 0.0087 (0.009203627343895376) [1m --> TIME: 2024-12-19 15:46:03 -- STEP: 3346/4076 -- GLOBAL_STEP: 19650[0m | > loss_text_ce: 0.035640597343444824 (0.03661182701968448) | > loss_mel_ce: 2.8923139572143555 (3.1441101063815546) | > loss: 0.7319886684417725 (0.7951804832897712) | > current_lr: 5e-06 | > step_time: 0.2326 (0.23021734145609696) | > loader_time: 0.0093 (0.009202172281638079) [1m --> TIME: 2024-12-19 15:46:24 -- STEP: 3396/4076 -- GLOBAL_STEP: 19700[0m | > loss_text_ce: 0.04067429155111313 (0.036608585067646304) | > loss_mel_ce: 3.1467747688293457 (3.144053580061707) | > loss: 0.7968622446060181 (0.7951655412801445) | > current_lr: 5e-06 | > step_time: 0.153 (0.2302080491827571) | > loader_time: 0.0085 (0.00920043150303082) [1m --> TIME: 2024-12-19 15:46:45 -- STEP: 3446/4076 -- GLOBAL_STEP: 19750[0m | > loss_text_ce: 0.03591074049472809 (0.03661437610459417) | > loss_mel_ce: 2.972672700881958 (3.1435918354116343) | > loss: 0.7521458864212036 (0.7950515528612199) | > current_lr: 5e-06 | > step_time: 0.2138 (0.23018037428775645) | > loader_time: 0.009 (0.009197676313248627) [1m --> TIME: 2024-12-19 15:47:06 -- STEP: 3496/4076 -- GLOBAL_STEP: 19800[0m | > loss_text_ce: 0.03733564913272858 (0.03660188577120843) | > loss_mel_ce: 3.023622989654541 (3.143282048898649) | > loss: 0.7652396559715271 (0.7949709836521467) | > current_lr: 5e-06 | > step_time: 0.2659 (0.23023268403123123) | > loader_time: 0.0098 (0.009198463208615388) [1m --> TIME: 2024-12-19 15:47:27 -- STEP: 3546/4076 -- GLOBAL_STEP: 19850[0m | > loss_text_ce: 0.03585575148463249 (0.03659868059082101) | > loss_mel_ce: 3.322780132293701 (3.1429516949091534) | > loss: 0.8396589756011963 (0.794887593891146) | > current_lr: 5e-06 | > step_time: 0.234 (0.23023095732009558) | > loader_time: 0.0091 (0.009199489258939928) [1m --> TIME: 2024-12-19 15:47:48 -- STEP: 3596/4076 -- GLOBAL_STEP: 19900[0m | > loss_text_ce: 0.03475547581911087 (0.036599094975839214) | > loss_mel_ce: 2.913578987121582 (3.1428951147666098) | > loss: 0.7370836138725281 (0.7948735524901295) | > current_lr: 5e-06 | > step_time: 0.1524 (0.23024067789879743) | > loader_time: 0.0093 (0.009197328724505786) [1m --> TIME: 2024-12-19 15:48:09 -- STEP: 3646/4076 -- GLOBAL_STEP: 19950[0m | > loss_text_ce: 0.03606134280562401 (0.03659837048202431) | > loss_mel_ce: 2.965160846710205 (3.14247386022642) | > loss: 0.7503055334091187 (0.7947680576940979) | > current_lr: 5e-06 | > step_time: 0.2308 (0.23022479188357228) | > loader_time: 0.0102 (0.009200606055790702) [1m --> TIME: 2024-12-19 15:48:29 -- STEP: 3696/4076 -- GLOBAL_STEP: 20000[0m | > loss_text_ce: 0.03792310133576393 (0.03659922098619989) | > loss_mel_ce: 3.0167715549468994 (3.141984056794282) | > loss: 0.7636736631393433 (0.7946458194098432) | > current_lr: 5e-06 | > step_time: 0.2343 (0.2300643276471597) | > loader_time: 0.0088 (0.009205339016852457) > CHECKPOINT : /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb/checkpoint_20000.pth [1m --> TIME: 2024-12-19 15:48:59 -- STEP: 3746/4076 -- GLOBAL_STEP: 20050[0m | > loss_text_ce: 0.03611037880182266 (0.036594718528112954) | > loss_mel_ce: 3.4521985054016113 (3.1419988784698387) | > loss: 0.872077226638794 (0.7946483992314632) | > current_lr: 5e-06 | > step_time: 0.209 (0.23002398892385853) | > loader_time: 0.009 (0.009210324007371387) [1m --> TIME: 2024-12-19 15:49:20 -- STEP: 3796/4076 -- GLOBAL_STEP: 20100[0m | > loss_text_ce: 0.03760936111211777 (0.03659696799915905) | > loss_mel_ce: 3.1912682056427 (3.141155857745915) | > loss: 0.807219386100769 (0.7944382064523386) | > current_lr: 5e-06 | > step_time: 0.2716 (0.23000267170000635) | > loader_time: 0.0085 (0.009208633349492014) [1m --> TIME: 2024-12-19 15:49:40 -- STEP: 3846/4076 -- GLOBAL_STEP: 20150[0m | > loss_text_ce: 0.037223830819129944 (0.03660239414634014) | > loss_mel_ce: 3.125035285949707 (3.1413449052095537) | > loss: 0.790564775466919 (0.794486824882198) | > current_lr: 5e-06 | > step_time: 0.232 (0.22987536843468984) | > loader_time: 0.0093 (0.009209682652557278) [1m --> TIME: 2024-12-19 15:50:01 -- STEP: 3896/4076 -- GLOBAL_STEP: 20200[0m | > loss_text_ce: 0.040632523596286774 (0.036592098644760616) | > loss_mel_ce: 2.8659167289733887 (3.141400429074035) | > loss: 0.7266373038291931 (0.7944981319336433) | > current_lr: 5e-06 | > step_time: 0.1738 (0.22983357971209037) | > loader_time: 0.0097 (0.0092117414337409) [1m --> TIME: 2024-12-19 15:50:22 -- STEP: 3946/4076 -- GLOBAL_STEP: 20250[0m | > loss_text_ce: 0.037289250642061234 (0.036590532900069805) | > loss_mel_ce: 3.069640874862671 (3.140870818316) | > loss: 0.7767325043678284 (0.7943653377916267) | > current_lr: 5e-06 | > step_time: 0.2113 (0.22987093559328947) | > loader_time: 0.0088 (0.009210210008423052) [1m --> TIME: 2024-12-19 15:50:44 -- STEP: 3996/4076 -- GLOBAL_STEP: 20300[0m | > loss_text_ce: 0.039547961205244064 (0.03657663267085313) | > loss_mel_ce: 3.358712673187256 (3.1403366472866683) | > loss: 0.8495651483535767 (0.7942283199922937) | > current_lr: 5e-06 | > step_time: 0.2413 (0.2299402699217544) | > loader_time: 0.0109 (0.009208891365501869) [1m --> TIME: 2024-12-19 15:51:05 -- STEP: 4046/4076 -- GLOBAL_STEP: 20350[0m | > loss_text_ce: 0.0431065708398819 (0.036570950855514386) | > loss_mel_ce: 2.9225263595581055 (3.1398072590406025) | > loss: 0.7414082288742065 (0.7940945524392716) | > current_lr: 5e-06 | > step_time: 0.2131 (0.22992932584341133) | > loader_time: 0.0101 (0.00920568647445929) [1m > EVALUATION [0m [1m--> EVAL PERFORMANCE[0m | > avg_loader_time:[91m 0.46630443976475644 [0m(+0.0006878651105440836) | > avg_loss_text_ce:[92m 0.03507091377217036 [0m(-0.00045434098977309434) | > avg_loss_mel_ce:[92m 3.1932988533606896 [0m(-0.008945079950185786) | > avg_loss:[92m 3.2283697770192075 [0m(-0.0093994140625) > BEST MODEL : /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb/best_model_20380.pth [4m[1m > EPOCH: 5/8[0m --> /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb [1m > TRAINING (2024-12-19 15:52:30) [0m [1m --> TIME: 2024-12-19 15:52:39 -- STEP: 20/4076 -- GLOBAL_STEP: 20400[0m | > loss_text_ce: 0.03891577571630478 (0.03686489844694733) | > loss_mel_ce: 2.954066276550293 (3.145647954940796) | > loss: 0.7482455372810364 (0.7956282138824463) | > current_lr: 5e-06 | > step_time: 0.2664 (0.2242372751235962) | > loader_time: 0.011 (0.009990942478179932) [1m --> TIME: 2024-12-19 15:53:00 -- STEP: 70/4076 -- GLOBAL_STEP: 20450[0m | > loss_text_ce: 0.03497224301099777 (0.03676195277699403) | > loss_mel_ce: 3.089182138442993 (3.130953083719526) | > loss: 0.7810385823249817 (0.7919287596430097) | > current_lr: 5e-06 | > step_time: 0.2324 (0.22653205054146902) | > loader_time: 0.0085 (0.009757259913853237) [1m --> TIME: 2024-12-19 15:53:21 -- STEP: 120/4076 -- GLOBAL_STEP: 20500[0m | > loss_text_ce: 0.03734421730041504 (0.03656642576679587) | > loss_mel_ce: 3.101015090942383 (3.1097387890021007) | > loss: 0.7845898270606995 (0.786576304336389) | > current_lr: 5e-06 | > step_time: 0.2064 (0.2254395802815755) | > loader_time: 0.0089 (0.009469137589136763) [1m --> TIME: 2024-12-19 15:53:41 -- STEP: 170/4076 -- GLOBAL_STEP: 20550[0m | > loss_text_ce: 0.03347207233309746 (0.03646243037108112) | > loss_mel_ce: 2.982952356338501 (3.10969040814568) | > loss: 0.7541061043739319 (0.7865382099852842) | > current_lr: 5e-06 | > step_time: 0.2495 (0.22308445117052864) | > loader_time: 0.0083 (0.009348782371072213) [1m --> TIME: 2024-12-19 15:54:01 -- STEP: 220/4076 -- GLOBAL_STEP: 20600[0m | > loss_text_ce: 0.036152228713035583 (0.03641957285898653) | > loss_mel_ce: 3.0168488025665283 (3.1069984186779367) | > loss: 0.7632502317428589 (0.7858544980937784) | > current_lr: 5e-06 | > step_time: 0.1781 (0.22303351705724542) | > loader_time: 0.0089 (0.009281055493788281) [1m --> TIME: 2024-12-19 15:54:22 -- STEP: 270/4076 -- GLOBAL_STEP: 20650[0m | > loss_text_ce: 0.04181690514087677 (0.036383924067572314) | > loss_mel_ce: 2.9529926776885986 (3.094160698078297) | > loss: 0.7487024068832397 (0.7826361550225152) | > current_lr: 5e-06 | > step_time: 0.2635 (0.22286729724318896) | > loader_time: 0.0084 (0.009223071734110511) [1m --> TIME: 2024-12-19 15:54:43 -- STEP: 320/4076 -- GLOBAL_STEP: 20700[0m | > loss_text_ce: 0.03755342215299606 (0.03634768868796527) | > loss_mel_ce: 2.9874095916748047 (3.0962966375052927) | > loss: 0.756240725517273 (0.7831610808148978) | > current_lr: 5e-06 | > step_time: 0.2446 (0.22371068745851516) | > loader_time: 0.011 (0.009210763871669757) [1m --> TIME: 2024-12-19 15:55:04 -- STEP: 370/4076 -- GLOBAL_STEP: 20750[0m | > loss_text_ce: 0.03443696349859238 (0.036380190613704795) | > loss_mel_ce: 3.0241785049438477 (3.102812276659785) | > loss: 0.7646538615226746 (0.7847981159751479) | > current_lr: 5e-06 | > step_time: 0.1824 (0.22508283950187064) | > loader_time: 0.0097 (0.009231803223893435) [1m --> TIME: 2024-12-19 15:55:25 -- STEP: 420/4076 -- GLOBAL_STEP: 20800[0m | > loss_text_ce: 0.03841005638241768 (0.03646010658482005) | > loss_mel_ce: 2.9890918731689453 (3.103707306725638) | > loss: 0.7568754553794861 (0.7850418525082724) | > current_lr: 5e-06 | > step_time: 0.1784 (0.22543599548793974) | > loader_time: 0.0084 (0.009183846768878732) [1m --> TIME: 2024-12-19 15:55:45 -- STEP: 470/4076 -- GLOBAL_STEP: 20850[0m | > loss_text_ce: 0.03502042964100838 (0.036394424799909016) | > loss_mel_ce: 3.4459381103515625 (3.102809346483109) | > loss: 0.8702396154403687 (0.7848009420202133) | > current_lr: 5e-06 | > step_time: 0.179 (0.2255118172219459) | > loader_time: 0.0077 (0.009173493689679058) [1m --> TIME: 2024-12-19 15:56:06 -- STEP: 520/4076 -- GLOBAL_STEP: 20900[0m | > loss_text_ce: 0.03192761912941933 (0.03632081396018086) | > loss_mel_ce: 3.0423717498779297 (3.1003907593396995) | > loss: 0.7685748338699341 (0.7841778922539491) | > current_lr: 5e-06 | > step_time: 0.1765 (0.22562417433812068) | > loader_time: 0.0087 (0.00915208688149083) [1m --> TIME: 2024-12-19 15:56:27 -- STEP: 570/4076 -- GLOBAL_STEP: 20950[0m | > loss_text_ce: 0.03248661383986473 (0.03628453251235843) | > loss_mel_ce: 3.125643491744995 (3.1019691609499747) | > loss: 0.7895325422286987 (0.7845634224122031) | > current_lr: 5e-06 | > step_time: 0.2285 (0.22607685808549852) | > loader_time: 0.0083 (0.009143910491675638) [1m --> TIME: 2024-12-19 15:56:48 -- STEP: 620/4076 -- GLOBAL_STEP: 21000[0m | > loss_text_ce: 0.039885494858026505 (0.03632781848371512) | > loss_mel_ce: 3.1439366340637207 (3.1022171481963126) | > loss: 0.7959555387496948 (0.7846362410053128) | > current_lr: 5e-06 | > step_time: 0.2174 (0.22649164892012077) | > loader_time: 0.0102 (0.009157874122742665) [1m --> TIME: 2024-12-19 15:57:09 -- STEP: 670/4076 -- GLOBAL_STEP: 21050[0m | > loss_text_ce: 0.034612271934747696 (0.03629925224596441) | > loss_mel_ce: 3.198854923248291 (3.1034820795059206) | > loss: 0.8083667755126953 (0.7849453325591872) | > current_lr: 5e-06 | > step_time: 0.2277 (0.22639699729520887) | > loader_time: 0.009 (0.00913409937673538) [1m --> TIME: 2024-12-19 15:57:30 -- STEP: 720/4076 -- GLOBAL_STEP: 21100[0m | > loss_text_ce: 0.0357612669467926 (0.03630075986600589) | > loss_mel_ce: 3.58170223236084 (3.1052355567614236) | > loss: 0.9043658971786499 (0.7853840788205466) | > current_lr: 5e-06 | > step_time: 0.1918 (0.22684214512507125) | > loader_time: 0.0099 (0.009119631515608875) [1m --> TIME: 2024-12-19 15:57:51 -- STEP: 770/4076 -- GLOBAL_STEP: 21150[0m | > loss_text_ce: 0.03359942138195038 (0.03630233431210771) | > loss_mel_ce: 3.467198610305786 (3.1066678332043933) | > loss: 0.8751994967460632 (0.7857425414896632) | > current_lr: 5e-06 | > step_time: 0.2135 (0.22688769055651384) | > loader_time: 0.009 (0.009114277208006206) [1m --> TIME: 2024-12-19 15:58:11 -- STEP: 820/4076 -- GLOBAL_STEP: 21200[0m | > loss_text_ce: 0.03716186434030533 (0.036296269377102994) | > loss_mel_ce: 3.14815616607666 (3.1063474800528548) | > loss: 0.7963294982910156 (0.7856609369196544) | > current_lr: 5e-06 | > step_time: 0.2332 (0.22679724199015924) | > loader_time: 0.0091 (0.009115256332769604) [1m --> TIME: 2024-12-19 15:58:33 -- STEP: 870/4076 -- GLOBAL_STEP: 21250[0m | > loss_text_ce: 0.03952629864215851 (0.03631975964867868) | > loss_mel_ce: 3.089630603790283 (3.106033517574442) | > loss: 0.7822892069816589 (0.7855883188631344) | > current_lr: 5e-06 | > step_time: 0.3252 (0.22734083087964993) | > loader_time: 0.0092 (0.009113233116851431) [1m --> TIME: 2024-12-19 15:58:53 -- STEP: 920/4076 -- GLOBAL_STEP: 21300[0m | > loss_text_ce: 0.03451930359005928 (0.03628819902913406) | > loss_mel_ce: 3.257795572280884 (3.107909708696863) | > loss: 0.8230786919593811 (0.7860494765250581) | > current_lr: 5e-06 | > step_time: 0.192 (0.2268421318220056) | > loader_time: 0.0079 (0.009108465909957862) [1m --> TIME: 2024-12-19 15:59:14 -- STEP: 970/4076 -- GLOBAL_STEP: 21350[0m | > loss_text_ce: 0.03865788131952286 (0.03630559805627034) | > loss_mel_ce: 2.961118221282959 (3.1091727802433917) | > loss: 0.7499440312385559 (0.7863695941020534) | > current_lr: 5e-06 | > step_time: 0.2313 (0.2272052489605147) | > loader_time: 0.0105 (0.009098039459936376) [1m --> TIME: 2024-12-19 15:59:35 -- STEP: 1020/4076 -- GLOBAL_STEP: 21400[0m | > loss_text_ce: 0.037110552191734314 (0.03629630284529987) | > loss_mel_ce: 3.1378185749053955 (3.1115685009488874) | > loss: 0.7937322854995728 (0.7869662005527348) | > current_lr: 5e-06 | > step_time: 0.1901 (0.22728728102702722) | > loader_time: 0.0102 (0.009088954738542125) [1m --> TIME: 2024-12-19 15:59:56 -- STEP: 1070/4076 -- GLOBAL_STEP: 21450[0m | > loss_text_ce: 0.03785227611660957 (0.036317834503530926) | > loss_mel_ce: 2.913205623626709 (3.1105719082823433) | > loss: 0.7377644777297974 (0.7867224353496162) | > current_lr: 5e-06 | > step_time: 0.214 (0.22728565496819048) | > loader_time: 0.0097 (0.009087685112641217) [1m --> TIME: 2024-12-19 16:00:17 -- STEP: 1120/4076 -- GLOBAL_STEP: 21500[0m | > loss_text_ce: 0.03277966380119324 (0.03630271252144927) | > loss_mel_ce: 3.005974531173706 (3.110018696103777) | > loss: 0.7596885561943054 (0.7865803519529959) | > current_lr: 5e-06 | > step_time: 0.2373 (0.2272405305079052) | > loader_time: 0.0089 (0.009084198517458763) [1m --> TIME: 2024-12-19 16:00:37 -- STEP: 1170/4076 -- GLOBAL_STEP: 21550[0m | > loss_text_ce: 0.03313708305358887 (0.03633305080012124) | > loss_mel_ce: 3.035693883895874 (3.110627474540319) | > loss: 0.7672077417373657 (0.7867401312049638) | > current_lr: 5e-06 | > step_time: 0.2079 (0.22714659495231432) | > loader_time: 0.0101 (0.009093051486545123) [1m --> TIME: 2024-12-19 16:00:59 -- STEP: 1220/4076 -- GLOBAL_STEP: 21600[0m | > loss_text_ce: 0.03496648371219635 (0.03632678728184251) | > loss_mel_ce: 2.8390746116638184 (3.1129217482004) | > loss: 0.7185102701187134 (0.7873121336835335) | > current_lr: 5e-06 | > step_time: 0.1945 (0.22759620596150884) | > loader_time: 0.0083 (0.009099108664715864) [1m --> TIME: 2024-12-19 16:01:20 -- STEP: 1270/4076 -- GLOBAL_STEP: 21650[0m | > loss_text_ce: 0.038044631481170654 (0.03633257557205327) | > loss_mel_ce: 3.444667339324951 (3.1144172448811562) | > loss: 0.8706780076026917 (0.7876874549651709) | > current_lr: 5e-06 | > step_time: 0.3189 (0.22780791549232063) | > loader_time: 0.0089 (0.009102500705268425) [1m --> TIME: 2024-12-19 16:01:41 -- STEP: 1320/4076 -- GLOBAL_STEP: 21700[0m | > loss_text_ce: 0.037776581943035126 (0.036329126355210045) | > loss_mel_ce: 3.136625289916992 (3.1153585237084) | > loss: 0.7936004400253296 (0.7879219123811431) | > current_lr: 5e-06 | > step_time: 0.1731 (0.22778855363527933) | > loader_time: 0.0083 (0.009093267267400559) [1m --> TIME: 2024-12-19 16:02:01 -- STEP: 1370/4076 -- GLOBAL_STEP: 21750[0m | > loss_text_ce: 0.03745720908045769 (0.036322866083823) | > loss_mel_ce: 3.106187582015991 (3.117033938421819) | > loss: 0.7859112024307251 (0.7883392009856925) | > current_lr: 5e-06 | > step_time: 0.2302 (0.22763993096177595) | > loader_time: 0.0083 (0.00909259771778634) [1m --> TIME: 2024-12-19 16:02:23 -- STEP: 1420/4076 -- GLOBAL_STEP: 21800[0m | > loss_text_ce: 0.0358286052942276 (0.03632418906814613) | > loss_mel_ce: 3.0405757427215576 (3.1178804191065494) | > loss: 0.769101083278656 (0.7885511519623473) | > current_lr: 5e-06 | > step_time: 0.1762 (0.22794597451115997) | > loader_time: 0.0098 (0.00910288834236036) [1m --> TIME: 2024-12-19 16:02:44 -- STEP: 1470/4076 -- GLOBAL_STEP: 21850[0m | > loss_text_ce: 0.04200133681297302 (0.036337566840760116) | > loss_mel_ce: 3.3360626697540283 (3.1178304607365397) | > loss: 0.8445159792900085 (0.7885420068591629) | > current_lr: 5e-06 | > step_time: 0.1824 (0.22799242224012103) | > loader_time: 0.0087 (0.00909668280153857) [1m --> TIME: 2024-12-19 16:03:04 -- STEP: 1520/4076 -- GLOBAL_STEP: 21900[0m | > loss_text_ce: 0.03164207562804222 (0.03632577988564179) | > loss_mel_ce: 2.76314640045166 (3.117538833304454) | > loss: 0.6986970901489258 (0.7884661531369935) | > current_lr: 5e-06 | > step_time: 0.1751 (0.22780116420043142) | > loader_time: 0.0089 (0.009094861777205204) [1m --> TIME: 2024-12-19 16:03:24 -- STEP: 1570/4076 -- GLOBAL_STEP: 21950[0m | > loss_text_ce: 0.03750757500529289 (0.0362972843597174) | > loss_mel_ce: 2.7656383514404297 (3.117291325520557) | > loss: 0.7007864713668823 (0.7883971522568136) | > current_lr: 5e-06 | > step_time: 0.2068 (0.22747159110512702) | > loader_time: 0.0097 (0.0090984497859979) [1m --> TIME: 2024-12-19 16:03:45 -- STEP: 1620/4076 -- GLOBAL_STEP: 22000[0m | > loss_text_ce: 0.03400665521621704 (0.03631576687028559) | > loss_mel_ce: 3.3483009338378906 (3.117503782260564) | > loss: 0.8455768823623657 (0.7884548870133764) | > current_lr: 5e-06 | > step_time: 0.2155 (0.2276386614199038) | > loader_time: 0.0094 (0.009091129715060008) [1m --> TIME: 2024-12-19 16:04:06 -- STEP: 1670/4076 -- GLOBAL_STEP: 22050[0m | > loss_text_ce: 0.03616092726588249 (0.036322807787198104) | > loss_mel_ce: 3.1290533542633057 (3.1164948392057124) | > loss: 0.7913035750389099 (0.788204411546627) | > current_lr: 5e-06 | > step_time: 0.2118 (0.22776848641698233) | > loader_time: 0.0084 (0.009094382331756773) [1m --> TIME: 2024-12-19 16:04:28 -- STEP: 1720/4076 -- GLOBAL_STEP: 22100[0m | > loss_text_ce: 0.036518845707178116 (0.0363081654609549) | > loss_mel_ce: 3.285109519958496 (3.117140597382256) | > loss: 0.8304070830345154 (0.7883621903699497) | > current_lr: 5e-06 | > step_time: 0.2361 (0.22803580663924994) | > loader_time: 0.0105 (0.009092217129330298) [1m --> TIME: 2024-12-19 16:04:49 -- STEP: 1770/4076 -- GLOBAL_STEP: 22150[0m | > loss_text_ce: 0.03767281398177147 (0.036302934386004466) | > loss_mel_ce: 3.061823844909668 (3.11820107934165) | > loss: 0.7748741507530212 (0.7886260031980309) | > current_lr: 5e-06 | > step_time: 0.1863 (0.22788493619800287) | > loader_time: 0.0084 (0.009099229042139423) [1m --> TIME: 2024-12-19 16:05:10 -- STEP: 1820/4076 -- GLOBAL_STEP: 22200[0m | > loss_text_ce: 0.03284776955842972 (0.03629393848477972) | > loss_mel_ce: 3.1942696571350098 (3.118831292482522) | > loss: 0.8067793846130371 (0.7887813075558169) | > current_lr: 5e-06 | > step_time: 0.2375 (0.22805435356203016) | > loader_time: 0.0092 (0.009107316588307468) [1m --> TIME: 2024-12-19 16:05:31 -- STEP: 1870/4076 -- GLOBAL_STEP: 22250[0m | > loss_text_ce: 0.039214134216308594 (0.03629024066130583) | > loss_mel_ce: 3.0032694339752197 (3.118074767347325) | > loss: 0.7606208920478821 (0.788591251813154) | > current_lr: 5e-06 | > step_time: 0.1877 (0.22805637420817493) | > loader_time: 0.0098 (0.009113501354972301) [1m --> TIME: 2024-12-19 16:05:52 -- STEP: 1920/4076 -- GLOBAL_STEP: 22300[0m | > loss_text_ce: 0.031187616288661957 (0.03626028352979723) | > loss_mel_ce: 2.944251298904419 (3.117076895510156) | > loss: 0.7438597083091736 (0.7883342946879565) | > current_lr: 5e-06 | > step_time: 0.1787 (0.22827940322458745) | > loader_time: 0.0086 (0.009115452443559957) [1m --> TIME: 2024-12-19 16:06:13 -- STEP: 1970/4076 -- GLOBAL_STEP: 22350[0m | > loss_text_ce: 0.03446749225258827 (0.03625942989049221) | > loss_mel_ce: 2.8561606407165527 (3.1159933764317307) | > loss: 0.7226570248603821 (0.788063201444403) | > current_lr: 5e-06 | > step_time: 0.2289 (0.22824535527205103) | > loader_time: 0.0101 (0.00912483876126671) [1m --> TIME: 2024-12-19 16:06:34 -- STEP: 2020/4076 -- GLOBAL_STEP: 22400[0m | > loss_text_ce: 0.036167073994874954 (0.036255993876643085) | > loss_mel_ce: 3.1202852725982666 (3.116036614687135) | > loss: 0.7891131043434143 (0.7880731520676376) | > current_lr: 5e-06 | > step_time: 0.2326 (0.228556232995326) | > loader_time: 0.0083 (0.009120806727078875) [1m --> TIME: 2024-12-19 16:06:55 -- STEP: 2070/4076 -- GLOBAL_STEP: 22450[0m | > loss_text_ce: 0.035646628588438034 (0.0362512795857473) | > loss_mel_ce: 3.1560511589050293 (3.1151718871029095) | > loss: 0.7979244589805603 (0.7878557916136755) | > current_lr: 5e-06 | > step_time: 0.1871 (0.2283646399272237) | > loader_time: 0.0094 (0.00912322122693637) [1m --> TIME: 2024-12-19 16:07:16 -- STEP: 2120/4076 -- GLOBAL_STEP: 22500[0m | > loss_text_ce: 0.03137173503637314 (0.036250260394101695) | > loss_mel_ce: 2.852699041366577 (3.114414969705185) | > loss: 0.7210177183151245 (0.7876663074740824) | > current_lr: 5e-06 | > step_time: 0.1742 (0.22854866149290554) | > loader_time: 0.0089 (0.009127045127580743) [1m --> TIME: 2024-12-19 16:07:37 -- STEP: 2170/4076 -- GLOBAL_STEP: 22550[0m | > loss_text_ce: 0.038086116313934326 (0.036239275613516427) | > loss_mel_ce: 2.8890068531036377 (3.11353600398736) | > loss: 0.7317732572555542 (0.7874438198206061) | > current_lr: 5e-06 | > step_time: 0.2326 (0.2284741687335177) | > loader_time: 0.01 (0.009124640715287022) [1m --> TIME: 2024-12-19 16:07:58 -- STEP: 2220/4076 -- GLOBAL_STEP: 22600[0m | > loss_text_ce: 0.034623369574546814 (0.03623864462532036) | > loss_mel_ce: 3.097993850708008 (3.11390610879606) | > loss: 0.783154308795929 (0.7875361883425495) | > current_lr: 5e-06 | > step_time: 0.1769 (0.22869220972061152) | > loader_time: 0.0103 (0.009126044608451216) [1m --> TIME: 2024-12-19 16:08:19 -- STEP: 2270/4076 -- GLOBAL_STEP: 22650[0m | > loss_text_ce: 0.03823435679078102 (0.036248458802929796) | > loss_mel_ce: 3.0557501316070557 (3.1139828297535246) | > loss: 0.773496150970459 (0.7875578221770633) | > current_lr: 5e-06 | > step_time: 0.1868 (0.22862053348104333) | > loader_time: 0.0089 (0.009125269247046649) [1m --> TIME: 2024-12-19 16:08:40 -- STEP: 2320/4076 -- GLOBAL_STEP: 22700[0m | > loss_text_ce: 0.03637407720088959 (0.03624904164482424) | > loss_mel_ce: 3.28227162361145 (3.1136494816377263) | > loss: 0.8296614289283752 (0.7874746309272174) | > current_lr: 5e-06 | > step_time: 0.1928 (0.2285683199249465) | > loader_time: 0.0104 (0.009123176439055077) [1m --> TIME: 2024-12-19 16:09:01 -- STEP: 2370/4076 -- GLOBAL_STEP: 22750[0m | > loss_text_ce: 0.036814745515584946 (0.03623829225651954) | > loss_mel_ce: 3.1443095207214355 (3.112750261242382) | > loss: 0.7952810525894165 (0.7872471384609797) | > current_lr: 5e-06 | > step_time: 0.2785 (0.22865704655144284) | > loader_time: 0.0088 (0.009120328718096895) [1m --> TIME: 2024-12-19 16:09:22 -- STEP: 2420/4076 -- GLOBAL_STEP: 22800[0m | > loss_text_ce: 0.036298878490924835 (0.03623582002483618) | > loss_mel_ce: 3.7659354209899902 (3.112914766851539) | > loss: 0.950558602809906 (0.7872876468522485) | > current_lr: 5e-06 | > step_time: 0.2322 (0.22865887436984986) | > loader_time: 0.0092 (0.00912198380005261) [1m --> TIME: 2024-12-19 16:09:43 -- STEP: 2470/4076 -- GLOBAL_STEP: 22850[0m | > loss_text_ce: 0.037029899656772614 (0.03622908879539023) | > loss_mel_ce: 2.9833271503448486 (3.112241296536531) | > loss: 0.755089282989502 (0.7871175965075548) | > current_lr: 5e-06 | > step_time: 0.1871 (0.22879662899835873) | > loader_time: 0.0117 (0.009123953633945482) [1m --> TIME: 2024-12-19 16:10:04 -- STEP: 2520/4076 -- GLOBAL_STEP: 22900[0m | > loss_text_ce: 0.03801887482404709 (0.03621810344343506) | > loss_mel_ce: 2.791670322418213 (3.11138919071546) | > loss: 0.7074223160743713 (0.7869018236796058) | > current_lr: 5e-06 | > step_time: 0.1572 (0.22872232066260445) | > loader_time: 0.0084 (0.009126494801233691) [1m --> TIME: 2024-12-19 16:10:25 -- STEP: 2570/4076 -- GLOBAL_STEP: 22950[0m | > loss_text_ce: 0.0356491282582283 (0.036215734423105606) | > loss_mel_ce: 3.1009438037872314 (3.1105220853122786) | > loss: 0.7841482162475586 (0.7866844549949066) | > current_lr: 5e-06 | > step_time: 0.2813 (0.22869763903116902) | > loader_time: 0.0115 (0.009127249049769302) [1m --> TIME: 2024-12-19 16:10:46 -- STEP: 2620/4076 -- GLOBAL_STEP: 23000[0m | > loss_text_ce: 0.04022447392344475 (0.03621186594770271) | > loss_mel_ce: 3.199587821960449 (3.1097606359547374) | > loss: 0.8099530935287476 (0.7864931255123996) | > current_lr: 5e-06 | > step_time: 0.2414 (0.22877337413889762) | > loader_time: 0.009 (0.009128696135892218) [1m --> TIME: 2024-12-19 16:11:07 -- STEP: 2670/4076 -- GLOBAL_STEP: 23050[0m | > loss_text_ce: 0.03203190490603447 (0.03619785002931568) | > loss_mel_ce: 3.0523626804351807 (3.1084027192119397) | > loss: 0.7710986733436584 (0.7861501423607157) | > current_lr: 5e-06 | > step_time: 0.1874 (0.22874792854437667) | > loader_time: 0.0078 (0.009130610866046577) [1m --> TIME: 2024-12-19 16:11:27 -- STEP: 2720/4076 -- GLOBAL_STEP: 23100[0m | > loss_text_ce: 0.035015854984521866 (0.036193733097410224) | > loss_mel_ce: 3.1109986305236816 (3.1081076998921042) | > loss: 0.7865036129951477 (0.7860753583557464) | > current_lr: 5e-06 | > step_time: 0.1773 (0.22860071720445857) | > loader_time: 0.0086 (0.009133369431776159) [1m --> TIME: 2024-12-19 16:11:48 -- STEP: 2770/4076 -- GLOBAL_STEP: 23150[0m | > loss_text_ce: 0.037119701504707336 (0.03619990416731865) | > loss_mel_ce: 2.8725481033325195 (3.1080476284887855) | > loss: 0.7274169325828552 (0.7860618832739681) | > current_lr: 5e-06 | > step_time: 0.185 (0.22858667545800604) | > loader_time: 0.0092 (0.009140422972531098) [1m --> TIME: 2024-12-19 16:12:09 -- STEP: 2820/4076 -- GLOBAL_STEP: 23200[0m | > loss_text_ce: 0.03253389149904251 (0.03618999775499109) | > loss_mel_ce: 3.0318262577056885 (3.1083083254225747) | > loss: 0.7660900354385376 (0.7861245809506016) | > current_lr: 5e-06 | > step_time: 0.241 (0.22868759649019715) | > loader_time: 0.0101 (0.009146749466023552) [1m --> TIME: 2024-12-19 16:12:30 -- STEP: 2870/4076 -- GLOBAL_STEP: 23250[0m | > loss_text_ce: 0.03188789263367653 (0.03618761132818486) | > loss_mel_ce: 3.010045051574707 (3.108713962890549) | > loss: 0.7604832649230957 (0.7862253937156357) | > current_lr: 5e-06 | > step_time: 0.2308 (0.2286012175191155) | > loader_time: 0.0084 (0.00914905112794883) [1m --> TIME: 2024-12-19 16:12:51 -- STEP: 2920/4076 -- GLOBAL_STEP: 23300[0m | > loss_text_ce: 0.03261388838291168 (0.03618647401310409) | > loss_mel_ce: 3.409341812133789 (3.108139712026678) | > loss: 0.8604889512062073 (0.786081546708329) | > current_lr: 5e-06 | > step_time: 0.2358 (0.2287260994519273) | > loader_time: 0.0092 (0.009153990957834948) [1m --> TIME: 2024-12-19 16:13:12 -- STEP: 2970/4076 -- GLOBAL_STEP: 23350[0m | > loss_text_ce: 0.04012279585003853 (0.036186018590548916) | > loss_mel_ce: 3.187169313430786 (3.107929189357697) | > loss: 0.8068230152130127 (0.7860288021741089) | > current_lr: 5e-06 | > step_time: 0.3186 (0.2288029360851455) | > loader_time: 0.0097 (0.009158043829278912) [1m --> TIME: 2024-12-19 16:13:33 -- STEP: 3020/4076 -- GLOBAL_STEP: 23400[0m | > loss_text_ce: 0.036413807421922684 (0.03618737875749149) | > loss_mel_ce: 3.1072704792022705 (3.1077341220236807) | > loss: 0.7859210968017578 (0.7859803753380742) | > current_lr: 5e-06 | > step_time: 0.1589 (0.22884302147176883) | > loader_time: 0.0088 (0.00915782404261709) [1m --> TIME: 2024-12-19 16:13:54 -- STEP: 3070/4076 -- GLOBAL_STEP: 23450[0m | > loss_text_ce: 0.0344555601477623 (0.03618034261979584) | > loss_mel_ce: 2.89509654045105 (3.107697428554203) | > loss: 0.7323880195617676 (0.7859694428863275) | > current_lr: 5e-06 | > step_time: 0.188 (0.22888237179684717) | > loader_time: 0.0081 (0.009157339680078352) [1m --> TIME: 2024-12-19 16:14:15 -- STEP: 3120/4076 -- GLOBAL_STEP: 23500[0m | > loss_text_ce: 0.03463943302631378 (0.03617404756936229) | > loss_mel_ce: 3.3257620334625244 (3.107982717492645) | > loss: 0.8401003479957581 (0.7860391913316187) | > current_lr: 5e-06 | > step_time: 0.1938 (0.22889251640209785) | > loader_time: 0.0092 (0.00916335544525047) [1m --> TIME: 2024-12-19 16:14:36 -- STEP: 3170/4076 -- GLOBAL_STEP: 23550[0m | > loss_text_ce: 0.03305286914110184 (0.036172299401405726) | > loss_mel_ce: 3.383662223815918 (3.108285077137153) | > loss: 0.854178786277771 (0.786114344254653) | > current_lr: 5e-06 | > step_time: 0.2663 (0.22877864927899574) | > loader_time: 0.009 (0.009166076356304165) [1m --> TIME: 2024-12-19 16:14:57 -- STEP: 3220/4076 -- GLOBAL_STEP: 23600[0m | > loss_text_ce: 0.03634677827358246 (0.03617084338474354) | > loss_mel_ce: 3.1852834224700928 (3.1078195249071805) | > loss: 0.8054075241088867 (0.785997592199663) | > current_lr: 5e-06 | > step_time: 0.1771 (0.22874828117974796) | > loader_time: 0.0094 (0.009164740209994094) [1m --> TIME: 2024-12-19 16:15:17 -- STEP: 3270/4076 -- GLOBAL_STEP: 23650[0m | > loss_text_ce: 0.03137487918138504 (0.03617337489107615) | > loss_mel_ce: 2.9052112102508545 (3.1070420722349006) | > loss: 0.7341465353965759 (0.7858038619023943) | > current_lr: 5e-06 | > step_time: 0.2493 (0.22871060437018717) | > loader_time: 0.0085 (0.00916495957505811) [1m --> TIME: 2024-12-19 16:15:38 -- STEP: 3320/4076 -- GLOBAL_STEP: 23700[0m | > loss_text_ce: 0.03442976251244545 (0.036164748633325354) | > loss_mel_ce: 2.7182676792144775 (3.106495648766142) | > loss: 0.6881743669509888 (0.7856650994484682) | > current_lr: 5e-06 | > step_time: 0.173 (0.2287028001733573) | > loader_time: 0.0105 (0.009170932152185072) [1m --> TIME: 2024-12-19 16:15:59 -- STEP: 3370/4076 -- GLOBAL_STEP: 23750[0m | > loss_text_ce: 0.03296506777405739 (0.036161408842560666) | > loss_mel_ce: 2.9507455825805664 (3.10573993967266) | > loss: 0.7459276914596558 (0.7854753372577247) | > current_lr: 5e-06 | > step_time: 0.1868 (0.22866014082990696) | > loader_time: 0.0099 (0.009170259707759297) [1m --> TIME: 2024-12-19 16:16:20 -- STEP: 3420/4076 -- GLOBAL_STEP: 23800[0m | > loss_text_ce: 0.033946964889764786 (0.036160041362439395) | > loss_mel_ce: 3.1020448207855225 (3.1062916279536275) | > loss: 0.7839979529380798 (0.785612917416974) | > current_lr: 5e-06 | > step_time: 0.1556 (0.228623040098893) | > loader_time: 0.0098 (0.009172636793370803) [1m --> TIME: 2024-12-19 16:16:41 -- STEP: 3470/4076 -- GLOBAL_STEP: 23850[0m | > loss_text_ce: 0.03680628165602684 (0.03615541667046745) | > loss_mel_ce: 3.1780283451080322 (3.1053768603877336) | > loss: 0.8037086725234985 (0.7853830693949883) | > current_lr: 5e-06 | > step_time: 0.2937 (0.22873628751001715) | > loader_time: 0.0105 (0.009174928335360188) [1m --> TIME: 2024-12-19 16:17:01 -- STEP: 3520/4076 -- GLOBAL_STEP: 23900[0m | > loss_text_ce: 0.032118164002895355 (0.03615339354940018) | > loss_mel_ce: 2.900702953338623 (3.1055476274002713) | > loss: 0.733205258846283 (0.7854252553798934) | > current_lr: 5e-06 | > step_time: 0.263 (0.22855660197409716) | > loader_time: 0.0101 (0.009174570644443662) [1m --> TIME: 2024-12-19 16:17:23 -- STEP: 3570/4076 -- GLOBAL_STEP: 23950[0m | > loss_text_ce: 0.03680998459458351 (0.036150819797288014) | > loss_mel_ce: 3.204160213470459 (3.105264690877346) | > loss: 0.8102425336837769 (0.7853538777647898) | > current_lr: 5e-06 | > step_time: 0.2812 (0.22871408849871125) | > loader_time: 0.0093 (0.009171951852258824) [1m --> TIME: 2024-12-19 16:17:44 -- STEP: 3620/4076 -- GLOBAL_STEP: 24000[0m | > loss_text_ce: 0.035380538552999496 (0.036150339600830826) | > loss_mel_ce: 3.186622142791748 (3.1053454344443883) | > loss: 0.8055006861686707 (0.7853739435830827) | > current_lr: 5e-06 | > step_time: 0.1738 (0.22878442409947433) | > loader_time: 0.0084 (0.009167423366841667) [1m --> TIME: 2024-12-19 16:18:06 -- STEP: 3670/4076 -- GLOBAL_STEP: 24050[0m | > loss_text_ce: 0.03197275474667549 (0.03615225467130506) | > loss_mel_ce: 3.132211923599243 (3.1055248473580623) | > loss: 0.791046142578125 (0.7854192755852472) | > current_lr: 5e-06 | > step_time: 0.318 (0.2288877365699581) | > loader_time: 0.0094 (0.00916305790155393) [1m --> TIME: 2024-12-19 16:18:26 -- STEP: 3720/4076 -- GLOBAL_STEP: 24100[0m | > loss_text_ce: 0.033845726400613785 (0.03615403162235379) | > loss_mel_ce: 3.274860382080078 (3.1054417367263536) | > loss: 0.8271765112876892 (0.7853989421359953) | > current_lr: 5e-06 | > step_time: 0.2158 (0.22888165295764965) | > loader_time: 0.0089 (0.009164914649019936) [1m --> TIME: 2024-12-19 16:18:47 -- STEP: 3770/4076 -- GLOBAL_STEP: 24150[0m | > loss_text_ce: 0.03755621612071991 (0.03615272565935039) | > loss_mel_ce: 2.828932046890259 (3.1062677861524857) | > loss: 0.7166220545768738 (0.7856051279315897) | > current_lr: 5e-06 | > step_time: 0.2081 (0.2288929872866967) | > loader_time: 0.0097 (0.00917075413923995) [1m --> TIME: 2024-12-19 16:19:08 -- STEP: 3820/4076 -- GLOBAL_STEP: 24200[0m | > loss_text_ce: 0.037692587822675705 (0.03615305285793563) | > loss_mel_ce: 2.6487810611724854 (3.106052282343363) | > loss: 0.6716184020042419 (0.7855513338019086) | > current_lr: 5e-06 | > step_time: 0.1756 (0.22885350519449923) | > loader_time: 0.0086 (0.009172764623352349) [1m --> TIME: 2024-12-19 16:19:29 -- STEP: 3870/4076 -- GLOBAL_STEP: 24250[0m | > loss_text_ce: 0.038578856736421585 (0.03615997315117329) | > loss_mel_ce: 3.3106770515441895 (3.1056012684676717) | > loss: 0.8373139500617981 (0.785440310412291) | > current_lr: 5e-06 | > step_time: 0.2586 (0.22887144883473715) | > loader_time: 0.0113 (0.00918019520219904) [1m --> TIME: 2024-12-19 16:19:51 -- STEP: 3920/4076 -- GLOBAL_STEP: 24300[0m | > loss_text_ce: 0.0352962352335453 (0.036161304495715564) | > loss_mel_ce: 2.7523109912872314 (3.105597060918811) | > loss: 0.696901798248291 (0.7854395913530368) | > current_lr: 5e-06 | > step_time: 0.265 (0.22901761635225645) | > loader_time: 0.009 (0.0091796743626497) [1m --> TIME: 2024-12-19 16:20:12 -- STEP: 3970/4076 -- GLOBAL_STEP: 24350[0m | > loss_text_ce: 0.03544865548610687 (0.03615977568716115) | > loss_mel_ce: 3.310826539993286 (3.105629778628989) | > loss: 0.8365687727928162 (0.78544738858713) | > current_lr: 5e-06 | > step_time: 0.2144 (0.22905548291482614) | > loader_time: 0.0085 (0.00917384702552774) [1m --> TIME: 2024-12-19 16:20:33 -- STEP: 4020/4076 -- GLOBAL_STEP: 24400[0m | > loss_text_ce: 0.03359777480363846 (0.03615031922004417) | > loss_mel_ce: 3.181631088256836 (3.1056982116912755) | > loss: 0.8038071990013123 (0.7854621327338526) | > current_lr: 5e-06 | > step_time: 0.1747 (0.22905946406558972) | > loader_time: 0.0092 (0.009172903601802975) [1m --> TIME: 2024-12-19 16:20:54 -- STEP: 4070/4076 -- GLOBAL_STEP: 24450[0m | > loss_text_ce: 0.03554125502705574 (0.03615112695385333) | > loss_mel_ce: 2.7412145137786865 (3.105550855852172) | > loss: 0.6941889524459839 (0.7854254956913228) | > current_lr: 5e-06 | > step_time: 0.2752 (0.2291572971484585) | > loader_time: 0.0091 (0.00917520962417563) [1m > EVALUATION [0m [1m--> EVAL PERFORMANCE[0m | > avg_loader_time:[91m 0.4674201195056622 [0m(+0.0011156797409057617) | > avg_loss_text_ce:[92m 0.03474839146320636 [0m(-0.0003225223089639989) | > avg_loss_mel_ce:[92m 3.1833458863771877 [0m(-0.009952966983501899) | > avg_loss:[92m 3.2180942480380716 [0m(-0.010275528981135817) > BEST MODEL : /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb/best_model_24456.pth [4m[1m > EPOCH: 6/8[0m --> /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb [1m > TRAINING (2024-12-19 16:22:10) [0m [1m --> TIME: 2024-12-19 16:22:29 -- STEP: 44/4076 -- GLOBAL_STEP: 24500[0m | > loss_text_ce: 0.03709089756011963 (0.036244996357709176) | > loss_mel_ce: 3.499387741088867 (3.090658973563801) | > loss: 0.8841196298599243 (0.7817259891466661) | > current_lr: 5e-06 | > step_time: 0.177 (0.22729162194512106) | > loader_time: 0.0084 (0.010110540823502972) [1m --> TIME: 2024-12-19 16:22:51 -- STEP: 94/4076 -- GLOBAL_STEP: 24550[0m | > loss_text_ce: 0.037847138941287994 (0.03619790871805968) | > loss_mel_ce: 3.1458020210266113 (3.096265379418718) | > loss: 0.7959122657775879 (0.7831158200477032) | > current_lr: 5e-06 | > step_time: 0.2142 (0.2346816265836675) | > loader_time: 0.0105 (0.00982863345044724) [1m --> TIME: 2024-12-19 16:23:12 -- STEP: 144/4076 -- GLOBAL_STEP: 24600[0m | > loss_text_ce: 0.03410649672150612 (0.0362299983519026) | > loss_mel_ce: 3.1268253326416016 (3.0883504152297974) | > loss: 0.7902329564094543 (0.7811451028618548) | > current_lr: 5e-06 | > step_time: 0.2363 (0.23402008910973868) | > loader_time: 0.0086 (0.00955273376570807) [1m --> TIME: 2024-12-19 16:23:33 -- STEP: 194/4076 -- GLOBAL_STEP: 24650[0m | > loss_text_ce: 0.03265712410211563 (0.03604448799850399) | > loss_mel_ce: 3.1171953678131104 (3.0897760809082344) | > loss: 0.7874631285667419 (0.7814551419818522) | > current_lr: 5e-06 | > step_time: 0.2467 (0.23540727993876664) | > loader_time: 0.0082 (0.00937685401169295) [1m --> TIME: 2024-12-19 16:23:54 -- STEP: 244/4076 -- GLOBAL_STEP: 24700[0m | > loss_text_ce: 0.038509074598550797 (0.03603051263900076) | > loss_mel_ce: 3.1221320629119873 (3.0861410097997695) | > loss: 0.7901602983474731 (0.7805428802967069) | > current_lr: 5e-06 | > step_time: 0.1559 (0.2334774859616014) | > loader_time: 0.009 (0.009344472259771633) [1m --> TIME: 2024-12-19 16:24:15 -- STEP: 294/4076 -- GLOBAL_STEP: 24750[0m | > loss_text_ce: 0.0362529531121254 (0.036074640685502354) | > loss_mel_ce: 2.834198236465454 (3.0869885496541762) | > loss: 0.7176128029823303 (0.7807657972079557) | > current_lr: 5e-06 | > step_time: 0.2093 (0.23309211990460246) | > loader_time: 0.0099 (0.009337730148211633) [1m --> TIME: 2024-12-19 16:24:37 -- STEP: 344/4076 -- GLOBAL_STEP: 24800[0m | > loss_text_ce: 0.035049162805080414 (0.03603511072457009) | > loss_mel_ce: 3.4314234256744385 (3.0859810931738045) | > loss: 0.8666181564331055 (0.7805040504696761) | > current_lr: 5e-06 | > step_time: 0.2362 (0.23312676022219103) | > loader_time: 0.0081 (0.009339045646578774) [1m --> TIME: 2024-12-19 16:24:57 -- STEP: 394/4076 -- GLOBAL_STEP: 24850[0m | > loss_text_ce: 0.03397591412067413 (0.03595560778707716) | > loss_mel_ce: 3.0586652755737305 (3.0864460135474436) | > loss: 0.7731602787971497 (0.7806004048301487) | > current_lr: 5e-06 | > step_time: 0.2138 (0.2326092120959674) | > loader_time: 0.0083 (0.009258373134632404) [1m --> TIME: 2024-12-19 16:25:18 -- STEP: 444/4076 -- GLOBAL_STEP: 24900[0m | > loss_text_ce: 0.03654526174068451 (0.03589714368436112) | > loss_mel_ce: 3.186527967453003 (3.084594236300873) | > loss: 0.8057683110237122 (0.7801228446197942) | > current_lr: 5e-06 | > step_time: 0.1905 (0.2324836109135602) | > loader_time: 0.0098 (0.009245132003818551) [1m --> TIME: 2024-12-19 16:25:39 -- STEP: 494/4076 -- GLOBAL_STEP: 24950[0m | > loss_text_ce: 0.03495066985487938 (0.03589298155701712) | > loss_mel_ce: 3.1896753311157227 (3.0836860945350253) | > loss: 0.8061565160751343 (0.7798947688297706) | > current_lr: 5e-06 | > step_time: 0.2132 (0.23219603179437429) | > loader_time: 0.009 (0.009196543500491003) [1m --> TIME: 2024-12-19 16:26:00 -- STEP: 544/4076 -- GLOBAL_STEP: 25000[0m | > loss_text_ce: 0.03695591539144516 (0.03587975441683628) | > loss_mel_ce: 2.540738105773926 (3.079223532448798) | > loss: 0.6444234848022461 (0.7787758214947057) | > current_lr: 5e-06 | > step_time: 0.1737 (0.2317362103392096) | > loader_time: 0.0092 (0.009152124909793634) [1m --> TIME: 2024-12-19 16:26:21 -- STEP: 594/4076 -- GLOBAL_STEP: 25050[0m | > loss_text_ce: 0.035244639962911606 (0.03589648778804323) | > loss_mel_ce: 3.54593825340271 (3.0791368195504862) | > loss: 0.8952957391738892 (0.7787583262631388) | > current_lr: 5e-06 | > step_time: 0.2145 (0.23173109610072692) | > loader_time: 0.0087 (0.00914266173687046) [1m --> TIME: 2024-12-19 16:26:42 -- STEP: 644/4076 -- GLOBAL_STEP: 25100[0m | > loss_text_ce: 0.036358583718538284 (0.03591222609306531) | > loss_mel_ce: 3.31634259223938 (3.080598052984443) | > loss: 0.8381752967834473 (0.7791275690246077) | > current_lr: 5e-06 | > step_time: 0.157 (0.23169260846902123) | > loader_time: 0.0084 (0.0091401083128793) [1m --> TIME: 2024-12-19 16:27:03 -- STEP: 694/4076 -- GLOBAL_STEP: 25150[0m | > loss_text_ce: 0.037113290280103683 (0.035924291114728106) | > loss_mel_ce: 3.275446653366089 (3.0839329449518997) | > loss: 0.8281399607658386 (0.779964308402037) | > current_lr: 5e-06 | > step_time: 0.2124 (0.23154190981422446) | > loader_time: 0.0082 (0.009105845555791952) [1m --> TIME: 2024-12-19 16:27:24 -- STEP: 744/4076 -- GLOBAL_STEP: 25200[0m | > loss_text_ce: 0.03492732346057892 (0.035951093740520926) | > loss_mel_ce: 3.115182399749756 (3.0842938490452343) | > loss: 0.7875274419784546 (0.78006123519072) | > current_lr: 5e-06 | > step_time: 0.1736 (0.23107826293155712) | > loader_time: 0.0085 (0.009105014224206261) [1m --> TIME: 2024-12-19 16:27:45 -- STEP: 794/4076 -- GLOBAL_STEP: 25250[0m | > loss_text_ce: 0.03736555203795433 (0.03594176724825232) | > loss_mel_ce: 3.023590326309204 (3.0811526934506044) | > loss: 0.7652389407157898 (0.7792736146251864) | > current_lr: 5e-06 | > step_time: 0.2745 (0.23095505423750023) | > loader_time: 0.0099 (0.009107499038542563) [1m --> TIME: 2024-12-19 16:28:06 -- STEP: 844/4076 -- GLOBAL_STEP: 25300[0m | > loss_text_ce: 0.03694440796971321 (0.03595338415098489) | > loss_mel_ce: 3.1679980754852295 (3.0818510338028493) | > loss: 0.8012356162071228 (0.779451103783897) | > current_lr: 5e-06 | > step_time: 0.2334 (0.2312285171300879) | > loader_time: 0.0085 (0.009088691376961815) [1m --> TIME: 2024-12-19 16:28:27 -- STEP: 894/4076 -- GLOBAL_STEP: 25350[0m | > loss_text_ce: 0.034063465893268585 (0.03596511734253678) | > loss_mel_ce: 2.9941306114196777 (3.0860870532541496) | > loss: 0.7570485472679138 (0.7805130421715297) | > current_lr: 5e-06 | > step_time: 0.279 (0.23078233950356775) | > loader_time: 0.0086 (0.009091103103603572) [1m --> TIME: 2024-12-19 16:28:47 -- STEP: 944/4076 -- GLOBAL_STEP: 25400[0m | > loss_text_ce: 0.03235286474227905 (0.03593846794091547) | > loss_mel_ce: 3.119389057159424 (3.088971538816471) | > loss: 0.7879354953765869 (0.7812275014565155) | > current_lr: 5e-06 | > step_time: 0.2163 (0.23038466022176257) | > loader_time: 0.0085 (0.009098671755548262) [1m --> TIME: 2024-12-19 16:29:08 -- STEP: 994/4076 -- GLOBAL_STEP: 25450[0m | > loss_text_ce: 0.032813046127557755 (0.035960667699170695) | > loss_mel_ce: 2.834540367126465 (3.087932920072163) | > loss: 0.7168383598327637 (0.7809733967905795) | > current_lr: 5e-06 | > step_time: 0.1789 (0.23051282793463357) | > loader_time: 0.0099 (0.00911521527848734) [1m --> TIME: 2024-12-19 16:29:29 -- STEP: 1044/4076 -- GLOBAL_STEP: 25500[0m | > loss_text_ce: 0.034374721348285675 (0.035985819799147604) | > loss_mel_ce: 3.3668439388275146 (3.0870006643949366) | > loss: 0.8503046631813049 (0.7807466208478046) | > current_lr: 5e-06 | > step_time: 0.1749 (0.23058285909594248) | > loader_time: 0.0085 (0.009121383286984041) [1m --> TIME: 2024-12-19 16:29:51 -- STEP: 1094/4076 -- GLOBAL_STEP: 25550[0m | > loss_text_ce: 0.03693421930074692 (0.035984889011646225) | > loss_mel_ce: 3.0531365871429443 (3.087924719728545) | > loss: 0.7725176811218262 (0.7809774020254286) | > current_lr: 5e-06 | > step_time: 0.2307 (0.2306979617863294) | > loader_time: 0.0084 (0.009129967070584768) [1m --> TIME: 2024-12-19 16:30:12 -- STEP: 1144/4076 -- GLOBAL_STEP: 25600[0m | > loss_text_ce: 0.03551695495843887 (0.035973805919537076) | > loss_mel_ce: 3.021191358566284 (3.08723209767075) | > loss: 0.7641770839691162 (0.780801475725391) | > current_lr: 5e-06 | > step_time: 0.157 (0.2307457502905305) | > loader_time: 0.0094 (0.009142556598970113) [1m --> TIME: 2024-12-19 16:30:32 -- STEP: 1194/4076 -- GLOBAL_STEP: 25650[0m | > loss_text_ce: 0.0347442664206028 (0.035975864957362866) | > loss_mel_ce: 2.9902517795562744 (3.086040952497192) | > loss: 0.7562490105628967 (0.7805042042205086) | > current_lr: 5e-06 | > step_time: 0.2311 (0.23048218310977542) | > loader_time: 0.0089 (0.00913930118982517) [1m --> TIME: 2024-12-19 16:30:53 -- STEP: 1244/4076 -- GLOBAL_STEP: 25700[0m | > loss_text_ce: 0.03352734073996544 (0.03596363575991323) | > loss_mel_ce: 3.3800649642944336 (3.08653845415238) | > loss: 0.8533980846405029 (0.780625522376257) | > current_lr: 5e-06 | > step_time: 0.1947 (0.23044144877283515) | > loader_time: 0.0105 (0.00914563626721742) [1m --> TIME: 2024-12-19 16:31:14 -- STEP: 1294/4076 -- GLOBAL_STEP: 25750[0m | > loss_text_ce: 0.03458752483129501 (0.03597517591401632) | > loss_mel_ce: 3.1557116508483887 (3.0870423331695505) | > loss: 0.7975748181343079 (0.7807543771989199) | > current_lr: 5e-06 | > step_time: 0.2131 (0.2303220894089806) | > loader_time: 0.0082 (0.009132465218097397) [1m --> TIME: 2024-12-19 16:31:35 -- STEP: 1344/4076 -- GLOBAL_STEP: 25800[0m | > loss_text_ce: 0.028837813064455986 (0.03595254433734918) | > loss_mel_ce: 2.909723997116089 (3.0865104430842956) | > loss: 0.7346404790878296 (0.7806157467532019) | > current_lr: 5e-06 | > step_time: 0.1783 (0.23010720312595362) | > loader_time: 0.01 (0.009133347088382367) [1m --> TIME: 2024-12-19 16:31:56 -- STEP: 1394/4076 -- GLOBAL_STEP: 25850[0m | > loss_text_ce: 0.03146497532725334 (0.03598357180408712) | > loss_mel_ce: 2.8815104961395264 (3.0857156872578284) | > loss: 0.728243887424469 (0.7804248145269701) | > current_lr: 5e-06 | > step_time: 0.2137 (0.23025066592600968) | > loader_time: 0.0101 (0.009146760662795838) [1m --> TIME: 2024-12-19 16:32:17 -- STEP: 1444/4076 -- GLOBAL_STEP: 25900[0m | > loss_text_ce: 0.032540157437324524 (0.03597640723165463) | > loss_mel_ce: 2.7634520530700684 (3.085226950718094) | > loss: 0.6989980340003967 (0.7803008391256151) | > current_lr: 5e-06 | > step_time: 0.1731 (0.2303680612439924) | > loader_time: 0.0103 (0.009153927132033255) [1m --> TIME: 2024-12-19 16:32:39 -- STEP: 1494/4076 -- GLOBAL_STEP: 25950[0m | > loss_text_ce: 0.033678408712148666 (0.03599060384165172) | > loss_mel_ce: 3.0233824253082275 (3.084619088345262) | > loss: 0.7642651796340942 (0.7801524227562362) | > current_lr: 5e-06 | > step_time: 0.2784 (0.23059425216761614) | > loader_time: 0.0085 (0.00914918578453013) [1m --> TIME: 2024-12-19 16:32:59 -- STEP: 1544/4076 -- GLOBAL_STEP: 26000[0m | > loss_text_ce: 0.03408481925725937 (0.03597580031081649) | > loss_mel_ce: 3.3152997493743896 (3.0844128618277407) | > loss: 0.8373461365699768 (0.7800971652559668) | > current_lr: 5e-06 | > step_time: 0.1912 (0.23031176634403086) | > loader_time: 0.0103 (0.009137857608844576) [1m --> TIME: 2024-12-19 16:33:20 -- STEP: 1594/4076 -- GLOBAL_STEP: 26050[0m | > loss_text_ce: 0.037803102284669876 (0.03597330989683026) | > loss_mel_ce: 2.6410787105560303 (3.0843618632559484) | > loss: 0.669720470905304 (0.7800837930203085) | > current_lr: 5e-06 | > step_time: 0.2139 (0.2302865798081478) | > loader_time: 0.0098 (0.009147271756994228) [1m --> TIME: 2024-12-19 16:33:40 -- STEP: 1644/4076 -- GLOBAL_STEP: 26100[0m | > loss_text_ce: 0.03395312651991844 (0.035982336671093886) | > loss_mel_ce: 2.8759679794311523 (3.0841971364624587) | > loss: 0.7274802923202515 (0.7800448679720982) | > current_lr: 5e-06 | > step_time: 0.1771 (0.23007838380887846) | > loader_time: 0.0085 (0.009148283016362621) [1m --> TIME: 2024-12-19 16:34:01 -- STEP: 1694/4076 -- GLOBAL_STEP: 26150[0m | > loss_text_ce: 0.03555075451731682 (0.035976514068499235) | > loss_mel_ce: 3.0616137981414795 (3.083289858864217) | > loss: 0.7742911577224731 (0.77981659298523) | > current_lr: 5e-06 | > step_time: 0.1879 (0.23004081975470744) | > loader_time: 0.0088 (0.009149964894426474) [1m --> TIME: 2024-12-19 16:34:22 -- STEP: 1744/4076 -- GLOBAL_STEP: 26200[0m | > loss_text_ce: 0.035925544798374176 (0.035976003868283216) | > loss_mel_ce: 3.1461989879608154 (3.0833415235948114) | > loss: 0.795531153678894 (0.7798293815635216) | > current_lr: 5e-06 | > step_time: 0.2407 (0.23001570589498638) | > loader_time: 0.0104 (0.009170858001490251) [1m --> TIME: 2024-12-19 16:34:43 -- STEP: 1794/4076 -- GLOBAL_STEP: 26250[0m | > loss_text_ce: 0.03750242292881012 (0.03597066640529942) | > loss_mel_ce: 2.9113929271698 (3.083293167377926) | > loss: 0.7372238636016846 (0.7798159582293288) | > current_lr: 5e-06 | > step_time: 0.2317 (0.23010865182780899) | > loader_time: 0.0102 (0.009189438793305702) [1m --> TIME: 2024-12-19 16:35:04 -- STEP: 1844/4076 -- GLOBAL_STEP: 26300[0m | > loss_text_ce: 0.03449835628271103 (0.03596878950756825) | > loss_mel_ce: 3.0575709342956543 (3.0834548522489955) | > loss: 0.7730173468589783 (0.7798559102926753) | > current_lr: 5e-06 | > step_time: 0.2351 (0.22997245703240016) | > loader_time: 0.0085 (0.009204185577897343) [1m --> TIME: 2024-12-19 16:35:24 -- STEP: 1894/4076 -- GLOBAL_STEP: 26350[0m | > loss_text_ce: 0.03530199080705643 (0.0359614714232707) | > loss_mel_ce: 3.077211618423462 (3.0821980177537935) | > loss: 0.7781283855438232 (0.7795398722480446) | > current_lr: 5e-06 | > step_time: 0.2104 (0.22973263314054782) | > loader_time: 0.008 (0.009205589072883066) [1m --> TIME: 2024-12-19 16:35:46 -- STEP: 1944/4076 -- GLOBAL_STEP: 26400[0m | > loss_text_ce: 0.03778392821550369 (0.035952111349705544) | > loss_mel_ce: 3.157726287841797 (3.0804020780840027) | > loss: 0.7988775372505188 (0.7790885472616543) | > current_lr: 5e-06 | > step_time: 0.1946 (0.22982903468756025) | > loader_time: 0.0085 (0.009202743263401612) [1m --> TIME: 2024-12-19 16:36:07 -- STEP: 1994/4076 -- GLOBAL_STEP: 26450[0m | > loss_text_ce: 0.03783620893955231 (0.035933370139625435) | > loss_mel_ce: 2.830636978149414 (3.079257313801029) | > loss: 0.7171183228492737 (0.7787976709447632) | > current_lr: 5e-06 | > step_time: 0.2295 (0.22977091519978482) | > loader_time: 0.0099 (0.009205792349582928) [1m --> TIME: 2024-12-19 16:36:27 -- STEP: 2044/4076 -- GLOBAL_STEP: 26500[0m | > loss_text_ce: 0.03635774552822113 (0.035921201055209684) | > loss_mel_ce: 3.2038137912750244 (3.0788580933895586) | > loss: 0.8100428581237793 (0.7786948235592499) | > current_lr: 5e-06 | > step_time: 0.2341 (0.22973113878832402) | > loader_time: 0.0087 (0.009206023584839882) [1m --> TIME: 2024-12-19 16:36:49 -- STEP: 2094/4076 -- GLOBAL_STEP: 26550[0m | > loss_text_ce: 0.04446883499622345 (0.03591822374811343) | > loss_mel_ce: 3.159742832183838 (3.0790263761649723) | > loss: 0.8010529279708862 (0.7787361499687095) | > current_lr: 5e-06 | > step_time: 0.1801 (0.22998805385378276) | > loader_time: 0.0083 (0.009207052624328088) [1m --> TIME: 2024-12-19 16:37:10 -- STEP: 2144/4076 -- GLOBAL_STEP: 26600[0m | > loss_text_ce: 0.035534586757421494 (0.03592542676974447) | > loss_mel_ce: 3.0341439247131348 (3.079318664189594) | > loss: 0.7674196362495422 (0.7788110226948761) | > current_lr: 5e-06 | > step_time: 0.1903 (0.22996874398259973) | > loader_time: 0.0088 (0.009204299369854715) [1m --> TIME: 2024-12-19 16:37:30 -- STEP: 2194/4076 -- GLOBAL_STEP: 26650[0m | > loss_text_ce: 0.033018264919519424 (0.03592002780899173) | > loss_mel_ce: 2.9296770095825195 (3.079715105002863) | > loss: 0.740673840045929 (0.7789087830996015) | > current_lr: 5e-06 | > step_time: 0.1861 (0.2297201476101453) | > loader_time: 0.0083 (0.00920160727383554) [1m --> TIME: 2024-12-19 16:37:52 -- STEP: 2244/4076 -- GLOBAL_STEP: 26700[0m | > loss_text_ce: 0.03713357076048851 (0.035929027214354155) | > loss_mel_ce: 3.268035888671875 (3.080587869765711) | > loss: 0.8262923359870911 (0.7791292241072275) | > current_lr: 5e-06 | > step_time: 0.2331 (0.22984964292954635) | > loader_time: 0.0097 (0.00919790765180945) [1m --> TIME: 2024-12-19 16:38:13 -- STEP: 2294/4076 -- GLOBAL_STEP: 26750[0m | > loss_text_ce: 0.033686958253383636 (0.0359177985118983) | > loss_mel_ce: 2.8740811347961426 (3.0800831867698593) | > loss: 0.7269420027732849 (0.7790002461974269) | > current_lr: 5e-06 | > step_time: 0.2105 (0.22979872083289787) | > loader_time: 0.0086 (0.009198028727417524) [1m --> TIME: 2024-12-19 16:38:34 -- STEP: 2344/4076 -- GLOBAL_STEP: 26800[0m | > loss_text_ce: 0.03096543438732624 (0.035896534730327466) | > loss_mel_ce: 3.381671190261841 (3.079869415166028) | > loss: 0.8531591296195984 (0.7789414873403903) | > current_lr: 5e-06 | > step_time: 0.2105 (0.229882049886033) | > loader_time: 0.011 (0.009198160606846464) [1m --> TIME: 2024-12-19 16:38:54 -- STEP: 2394/4076 -- GLOBAL_STEP: 26850[0m | > loss_text_ce: 0.03419956937432289 (0.03588162726935065) | > loss_mel_ce: 3.0597550868988037 (3.0800184998396727) | > loss: 0.7734886407852173 (0.7789750315541514) | > current_lr: 5e-06 | > step_time: 0.2077 (0.2297796961060144) | > loader_time: 0.0102 (0.009195976189602982) [1m --> TIME: 2024-12-19 16:39:15 -- STEP: 2444/4076 -- GLOBAL_STEP: 26900[0m | > loss_text_ce: 0.03677841275930405 (0.0358800731032306) | > loss_mel_ce: 3.0822627544403076 (3.0805754533766128) | > loss: 0.7797603011131287 (0.7791138814972586) | > current_lr: 5e-06 | > step_time: 0.1563 (0.22972010551614963) | > loader_time: 0.0102 (0.009199090070303057) [1m --> TIME: 2024-12-19 16:39:36 -- STEP: 2494/4076 -- GLOBAL_STEP: 26950[0m | > loss_text_ce: 0.03582632169127464 (0.03586974936001952) | > loss_mel_ce: 3.06988263130188 (3.0809154583151654) | > loss: 0.7764272093772888 (0.779196301841124) | > current_lr: 5e-06 | > step_time: 0.2326 (0.22975954611777494) | > loader_time: 0.0094 (0.009204403629471223) [1m --> TIME: 2024-12-19 16:39:57 -- STEP: 2544/4076 -- GLOBAL_STEP: 27000[0m | > loss_text_ce: 0.032229214906692505 (0.03587388594227746) | > loss_mel_ce: 3.126591920852661 (3.0804376548753574) | > loss: 0.7897052764892578 (0.7790778850984276) | > current_lr: 5e-06 | > step_time: 0.2095 (0.229858712861373) | > loader_time: 0.0094 (0.009206081617553277) [1m --> TIME: 2024-12-19 16:40:19 -- STEP: 2594/4076 -- GLOBAL_STEP: 27050[0m | > loss_text_ce: 0.037008944898843765 (0.035888373527213455) | > loss_mel_ce: 3.072999954223633 (3.079717784177914) | > loss: 0.7775022387504578 (0.7789015392661554) | > current_lr: 5e-06 | > step_time: 0.2918 (0.22989385534270323) | > loader_time: 0.0088 (0.009208144641602689) [1m --> TIME: 2024-12-19 16:40:39 -- STEP: 2644/4076 -- GLOBAL_STEP: 27100[0m | > loss_text_ce: 0.03650964796543121 (0.0358825363352636) | > loss_mel_ce: 2.8466928005218506 (3.0795011471692044) | > loss: 0.7208006381988525 (0.7788459206628363) | > current_lr: 5e-06 | > step_time: 0.1929 (0.22979907630611657) | > loader_time: 0.0095 (0.0092086177009317) [1m --> TIME: 2024-12-19 16:41:00 -- STEP: 2694/4076 -- GLOBAL_STEP: 27150[0m | > loss_text_ce: 0.03561297804117203 (0.03588850774512576) | > loss_mel_ce: 3.0611746311187744 (3.0796127490140583) | > loss: 0.7741969227790833 (0.7788753140477133) | > current_lr: 5e-06 | > step_time: 0.208 (0.2297556023289738) | > loader_time: 0.0079 (0.009208802922000336) [1m --> TIME: 2024-12-19 16:41:21 -- STEP: 2744/4076 -- GLOBAL_STEP: 27200[0m | > loss_text_ce: 0.0280907079577446 (0.035883387339602904) | > loss_mel_ce: 3.1368377208709717 (3.079131871114314) | > loss: 0.7912321090698242 (0.7787538145617212) | > current_lr: 5e-06 | > step_time: 0.1742 (0.2296235804655114) | > loader_time: 0.0083 (0.009196694755693222) [1m --> TIME: 2024-12-19 16:41:41 -- STEP: 2794/4076 -- GLOBAL_STEP: 27250[0m | > loss_text_ce: 0.03553541377186775 (0.03588783808663207) | > loss_mel_ce: 3.1325924396514893 (3.0794005200277454) | > loss: 0.7920319437980652 (0.7788220895514284) | > current_lr: 5e-06 | > step_time: 0.187 (0.22954465588588072) | > loader_time: 0.0083 (0.009188959832351892) [1m --> TIME: 2024-12-19 16:42:02 -- STEP: 2844/4076 -- GLOBAL_STEP: 27300[0m | > loss_text_ce: 0.03610730543732643 (0.03588013493674705) | > loss_mel_ce: 3.273090124130249 (3.0784749537915905) | > loss: 0.8272993564605713 (0.7785887722392287) | > current_lr: 5e-06 | > step_time: 0.1934 (0.229531050650547) | > loader_time: 0.0086 (0.009186330140056154) [1m --> TIME: 2024-12-19 16:42:23 -- STEP: 2894/4076 -- GLOBAL_STEP: 27350[0m | > loss_text_ce: 0.03666522726416588 (0.03586660754373249) | > loss_mel_ce: 3.101840019226074 (3.0783701567461956) | > loss: 0.784626305103302 (0.7785591911511659) | > current_lr: 5e-06 | > step_time: 0.2753 (0.22961189858898592) | > loader_time: 0.0093 (0.009181360170276069) [1m --> TIME: 2024-12-19 16:42:44 -- STEP: 2944/4076 -- GLOBAL_STEP: 27400[0m | > loss_text_ce: 0.03515461087226868 (0.03586770793797622) | > loss_mel_ce: 2.709073781967163 (3.0775614627029535) | > loss: 0.6860570907592773 (0.7783572927684237) | > current_lr: 5e-06 | > step_time: 0.1553 (0.22946337197461855) | > loader_time: 0.0087 (0.00918303932184759) [1m --> TIME: 2024-12-19 16:43:05 -- STEP: 2994/4076 -- GLOBAL_STEP: 27450[0m | > loss_text_ce: 0.03403496369719505 (0.035873267000184145) | > loss_mel_ce: 3.132742166519165 (3.07782350091354) | > loss: 0.7916942834854126 (0.7784241920681736) | > current_lr: 5e-06 | > step_time: 0.2326 (0.22961398961150653) | > loader_time: 0.0111 (0.009185263293539607) [1m --> TIME: 2024-12-19 16:43:26 -- STEP: 3044/4076 -- GLOBAL_STEP: 27500[0m | > loss_text_ce: 0.037480466067790985 (0.035869499954239735) | > loss_mel_ce: 3.052687644958496 (3.0776400763477776) | > loss: 0.7725419998168945 (0.778377394168668) | > current_lr: 5e-06 | > step_time: 0.1781 (0.22953470072827734) | > loader_time: 0.0083 (0.009188477689113957) [1m --> TIME: 2024-12-19 16:43:47 -- STEP: 3094/4076 -- GLOBAL_STEP: 27550[0m | > loss_text_ce: 0.033190954476594925 (0.03587254186286703) | > loss_mel_ce: 2.8225433826446533 (3.0775081859377744) | > loss: 0.7139335870742798 (0.7783451819504622) | > current_lr: 5e-06 | > step_time: 0.1801 (0.22944316042571508) | > loader_time: 0.0086 (0.009188077106118289) [1m --> TIME: 2024-12-19 16:44:07 -- STEP: 3144/4076 -- GLOBAL_STEP: 27600[0m | > loss_text_ce: 0.03621751070022583 (0.035873395607643396) | > loss_mel_ce: 2.9127063751220703 (3.0779394502099793) | > loss: 0.7372309565544128 (0.7784532114256729) | > current_lr: 5e-06 | > step_time: 0.1508 (0.22940744364837956) | > loader_time: 0.01 (0.009190636736746053) [1m --> TIME: 2024-12-19 16:44:28 -- STEP: 3194/4076 -- GLOBAL_STEP: 27650[0m | > loss_text_ce: 0.03816268593072891 (0.035878569374657755) | > loss_mel_ce: 3.0850889682769775 (3.0774277888317694) | > loss: 0.7808129191398621 (0.7783265894252055) | > current_lr: 5e-06 | > step_time: 0.1827 (0.2292811677988873) | > loader_time: 0.0095 (0.009186064965589094) [1m --> TIME: 2024-12-19 16:44:48 -- STEP: 3244/4076 -- GLOBAL_STEP: 27700[0m | > loss_text_ce: 0.039306048303842545 (0.03586988725140251) | > loss_mel_ce: 2.8663721084594727 (3.077365366436546) | > loss: 0.7264195680618286 (0.7783088133418895) | > current_lr: 5e-06 | > step_time: 0.1938 (0.22928663290825785) | > loader_time: 0.0085 (0.009189126450389306) [1m --> TIME: 2024-12-19 16:45:10 -- STEP: 3294/4076 -- GLOBAL_STEP: 27750[0m | > loss_text_ce: 0.03299015387892723 (0.03586817304386072) | > loss_mel_ce: 3.0218443870544434 (3.076715582851644) | > loss: 0.7637086510658264 (0.7781459388764754) | > current_lr: 5e-06 | > step_time: 0.2079 (0.22931851404105524) | > loader_time: 0.0089 (0.009193956671010112) [1m --> TIME: 2024-12-19 16:45:31 -- STEP: 3344/4076 -- GLOBAL_STEP: 27800[0m | > loss_text_ce: 0.030406763777136803 (0.0358618469039898) | > loss_mel_ce: 3.1926302909851074 (3.076382069425145) | > loss: 0.8057592511177063 (0.7780609790004038) | > current_lr: 5e-06 | > step_time: 0.1946 (0.22935877284079648) | > loader_time: 0.009 (0.009189750660549532) [1m --> TIME: 2024-12-19 16:45:52 -- STEP: 3394/4076 -- GLOBAL_STEP: 27850[0m | > loss_text_ce: 0.03510121628642082 (0.03585205164682589) | > loss_mel_ce: 2.9582102298736572 (3.0760732646120488) | > loss: 0.7483278512954712 (0.7779813289607208) | > current_lr: 5e-06 | > step_time: 0.2789 (0.22942492796100444) | > loader_time: 0.0083 (0.009191938829056731) [1m --> TIME: 2024-12-19 16:46:14 -- STEP: 3444/4076 -- GLOBAL_STEP: 27900[0m | > loss_text_ce: 0.03853924944996834 (0.03584348860561316) | > loss_mel_ce: 3.0825836658477783 (3.075613103677167) | > loss: 0.7802807092666626 (0.7778641479944377) | > current_lr: 5e-06 | > step_time: 0.1895 (0.22956665107735913) | > loader_time: 0.0104 (0.00918836606088833) [1m --> TIME: 2024-12-19 16:46:35 -- STEP: 3494/4076 -- GLOBAL_STEP: 27950[0m | > loss_text_ce: 0.03545915707945824 (0.03584579429757159) | > loss_mel_ce: 3.1979336738586426 (3.075522906041919) | > loss: 0.8083481788635254 (0.7778421749915811) | > current_lr: 5e-06 | > step_time: 0.2779 (0.2295981966978356) | > loader_time: 0.0103 (0.009192005797802714) [1m --> TIME: 2024-12-19 16:46:55 -- STEP: 3544/4076 -- GLOBAL_STEP: 28000[0m | > loss_text_ce: 0.038205698132514954 (0.03583623275989118) | > loss_mel_ce: 3.0666511058807373 (3.074960552475518) | > loss: 0.7762141823768616 (0.7776991962108329) | > current_lr: 5e-06 | > step_time: 0.1927 (0.22953969603467472) | > loader_time: 0.0105 (0.009193732022700973) [1m --> TIME: 2024-12-19 16:47:16 -- STEP: 3594/4076 -- GLOBAL_STEP: 28050[0m | > loss_text_ce: 0.03896123915910721 (0.03583750361498956) | > loss_mel_ce: 2.603925943374634 (3.0747361198424192) | > loss: 0.6607217788696289 (0.7776434057178134) | > current_lr: 5e-06 | > step_time: 0.1884 (0.22951518925415787) | > loader_time: 0.0104 (0.009194406921756116) [1m --> TIME: 2024-12-19 16:47:37 -- STEP: 3644/4076 -- GLOBAL_STEP: 28100[0m | > loss_text_ce: 0.030702970921993256 (0.03583738011380617) | > loss_mel_ce: 2.8662374019622803 (3.074461930800475) | > loss: 0.7242351174354553 (0.7775748276389912) | > current_lr: 5e-06 | > step_time: 0.1731 (0.22938833644963252) | > loader_time: 0.0085 (0.009190363300357012) [1m --> TIME: 2024-12-19 16:47:57 -- STEP: 3694/4076 -- GLOBAL_STEP: 28150[0m | > loss_text_ce: 0.034825850278139114 (0.03583858340361457) | > loss_mel_ce: 3.2461163997650146 (3.0741626976887817) | > loss: 0.820235550403595 (0.7775003201553609) | > current_lr: 5e-06 | > step_time: 0.2914 (0.22927001843145234) | > loader_time: 0.0095 (0.009187252830026237) [1m --> TIME: 2024-12-19 16:48:18 -- STEP: 3744/4076 -- GLOBAL_STEP: 28200[0m | > loss_text_ce: 0.035916898399591446 (0.035838077563203605) | > loss_mel_ce: 3.378948450088501 (3.073947845106447) | > loss: 0.8537163138389587 (0.7774464805634356) | > current_lr: 5e-06 | > step_time: 0.2412 (0.2292718747869516) | > loader_time: 0.0128 (0.009180939286692556) [1m --> TIME: 2024-12-19 16:48:39 -- STEP: 3794/4076 -- GLOBAL_STEP: 28250[0m | > loss_text_ce: 0.037220291793346405 (0.03583523161605246) | > loss_mel_ce: 3.426567792892456 (3.0742826847007003) | > loss: 0.8659470081329346 (0.7775294789686035) | > current_lr: 5e-06 | > step_time: 0.3183 (0.22933838759589711) | > loader_time: 0.0087 (0.00917966339421011) [1m --> TIME: 2024-12-19 16:49:01 -- STEP: 3844/4076 -- GLOBAL_STEP: 28300[0m | > loss_text_ce: 0.03329327702522278 (0.03582906230177426) | > loss_mel_ce: 3.1203572750091553 (3.073943313153052) | > loss: 0.7884126305580139 (0.7774430937778191) | > current_lr: 5e-06 | > step_time: 0.177 (0.22945193259450572) | > loader_time: 0.0101 (0.0091910204083565) [1m --> TIME: 2024-12-19 16:49:22 -- STEP: 3894/4076 -- GLOBAL_STEP: 28350[0m | > loss_text_ce: 0.033111024647951126 (0.03582746275276361) | > loss_mel_ce: 2.8269433975219727 (3.074021323742346) | > loss: 0.7150136232376099 (0.7774621965368402) | > current_lr: 5e-06 | > step_time: 0.1816 (0.2295068615696402) | > loader_time: 0.0112 (0.009190964466128304) [1m --> TIME: 2024-12-19 16:49:43 -- STEP: 3944/4076 -- GLOBAL_STEP: 28400[0m | > loss_text_ce: 0.03441106155514717 (0.035815340195240485) | > loss_mel_ce: 2.9212286472320557 (3.074571128851498) | > loss: 0.738909900188446 (0.7775966171741967) | > current_lr: 5e-06 | > step_time: 0.194 (0.2296525332434424) | > loader_time: 0.0085 (0.009192236840845872) [1m --> TIME: 2024-12-19 16:50:04 -- STEP: 3994/4076 -- GLOBAL_STEP: 28450[0m | > loss_text_ce: 0.03876904025673866 (0.03581271547512632) | > loss_mel_ce: 2.9365394115448 (3.074182816487759) | > loss: 0.7438271045684814 (0.7774988829224001) | > current_lr: 5e-06 | > step_time: 0.231 (0.22964575455435884) | > loader_time: 0.0084 (0.009191064877574566) [1m --> TIME: 2024-12-19 16:50:26 -- STEP: 4044/4076 -- GLOBAL_STEP: 28500[0m | > loss_text_ce: 0.037406936287879944 (0.03580899690356192) | > loss_mel_ce: 3.017991781234741 (3.0735873217988283) | > loss: 0.763849675655365 (0.7773490795602193) | > current_lr: 5e-06 | > step_time: 0.2421 (0.2296866828209097) | > loader_time: 0.0097 (0.009190954511172718) [1m > EVALUATION [0m [1m--> EVAL PERFORMANCE[0m | > avg_loader_time:[91m 0.4713844977892362 [0m(+0.003964378283573988) | > avg_loss_text_ce:[92m 0.034445052679914705 [0m(-0.000303338783291654) | > avg_loss_mel_ce:[92m 3.175478825202355 [0m(-0.007867061174832823) | > avg_loss:[92m 3.20992388175084 [0m(-0.008170366287231445) > BEST MODEL : /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb/best_model_28532.pth [4m[1m > EPOCH: 7/8[0m --> /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb [1m > TRAINING (2024-12-19 16:51:52) [0m [1m --> TIME: 2024-12-19 16:52:00 -- STEP: 18/4076 -- GLOBAL_STEP: 28550[0m | > loss_text_ce: 0.04150627925992012 (0.0366192486964994) | > loss_mel_ce: 2.7658679485321045 (3.054669155014886) | > loss: 0.7018435597419739 (0.7728220886654324) | > current_lr: 5e-06 | > step_time: 0.2917 (0.22500240802764893) | > loader_time: 0.0107 (0.009970148404439291) [1m --> TIME: 2024-12-19 16:52:22 -- STEP: 68/4076 -- GLOBAL_STEP: 28600[0m | > loss_text_ce: 0.035407815128564835 (0.035937802318264454) | > loss_mel_ce: 2.9921371936798096 (3.0689544046626374) | > loss: 0.7568862438201904 (0.7762230503208497) | > current_lr: 5e-06 | > step_time: 0.1749 (0.23526298298555262) | > loader_time: 0.0102 (0.01036607167300056) [1m --> TIME: 2024-12-19 16:52:42 -- STEP: 118/4076 -- GLOBAL_STEP: 28650[0m | > loss_text_ce: 0.03384576737880707 (0.03576444630842592) | > loss_mel_ce: 2.8103153705596924 (3.0657725253347623) | > loss: 0.7110402584075928 (0.7753842422517679) | > current_lr: 5e-06 | > step_time: 0.1808 (0.2299226178961285) | > loader_time: 0.0079 (0.00993387173798125) [1m --> TIME: 2024-12-19 16:53:04 -- STEP: 168/4076 -- GLOBAL_STEP: 28700[0m | > loss_text_ce: 0.036302562803030014 (0.03579983961147566) | > loss_mel_ce: 3.1203360557556152 (3.0675512864476144) | > loss: 0.7891596555709839 (0.7758377808190527) | > current_lr: 5e-06 | > step_time: 0.1556 (0.2333331931205023) | > loader_time: 0.0084 (0.009624598991303215) [1m --> TIME: 2024-12-19 16:53:25 -- STEP: 218/4076 -- GLOBAL_STEP: 28750[0m | > loss_text_ce: 0.034882426261901855 (0.03581682101336366) | > loss_mel_ce: 3.182070255279541 (3.0699301170646587) | > loss: 0.8042382001876831 (0.7764367346369893) | > current_lr: 5e-06 | > step_time: 0.2321 (0.23296115813999002) | > loader_time: 0.0098 (0.0094977157925247) [1m --> TIME: 2024-12-19 16:53:46 -- STEP: 268/4076 -- GLOBAL_STEP: 28800[0m | > loss_text_ce: 0.034351542592048645 (0.03567007127037243) | > loss_mel_ce: 2.708862066268921 (3.067293725796599) | > loss: 0.6858034133911133 (0.7757409490756134) | > current_lr: 5e-06 | > step_time: 0.1744 (0.2321814609997308) | > loader_time: 0.0107 (0.009452529807588945) [1m --> TIME: 2024-12-19 16:54:07 -- STEP: 318/4076 -- GLOBAL_STEP: 28850[0m | > loss_text_ce: 0.03183898702263832 (0.035677587531460164) | > loss_mel_ce: 3.2673606872558594 (3.064761980524601) | > loss: 0.8247998952865601 (0.7751098916215718) | > current_lr: 5e-06 | > step_time: 0.2144 (0.23137789702265518) | > loader_time: 0.0085 (0.009417662080728783) [1m --> TIME: 2024-12-19 16:54:28 -- STEP: 368/4076 -- GLOBAL_STEP: 28900[0m | > loss_text_ce: 0.03536034747958183 (0.035683411983368205) | > loss_mel_ce: 2.994971990585327 (3.067609971632127) | > loss: 0.7575830817222595 (0.7758233461367048) | > current_lr: 5e-06 | > step_time: 0.1911 (0.23152492422124613) | > loader_time: 0.0087 (0.009385596150937285) [1m --> TIME: 2024-12-19 16:54:49 -- STEP: 418/4076 -- GLOBAL_STEP: 28950[0m | > loss_text_ce: 0.035488590598106384 (0.03565303403349442) | > loss_mel_ce: 3.0430052280426025 (3.0652501292205874) | > loss: 0.7696234583854675 (0.7752257913493654) | > current_lr: 5e-06 | > step_time: 0.2346 (0.23079753948740983) | > loader_time: 0.0086 (0.009360240406967231) [1m --> TIME: 2024-12-19 16:55:10 -- STEP: 468/4076 -- GLOBAL_STEP: 29000[0m | > loss_text_ce: 0.031796637922525406 (0.035607930368337884) | > loss_mel_ce: 3.193532943725586 (3.064708064763973) | > loss: 0.8063324093818665 (0.7750789994357997) | > current_lr: 5e-06 | > step_time: 0.1915 (0.23082408640119764) | > loader_time: 0.0085 (0.00931098726060655) [1m --> TIME: 2024-12-19 16:55:31 -- STEP: 518/4076 -- GLOBAL_STEP: 29050[0m | > loss_text_ce: 0.03604936599731445 (0.035605484155873314) | > loss_mel_ce: 2.8933019638061523 (3.0613070238511066) | > loss: 0.7323378324508667 (0.7742281275366264) | > current_lr: 5e-06 | > step_time: 0.2497 (0.2311122665994416) | > loader_time: 0.0084 (0.009294721610757832) [1m --> TIME: 2024-12-19 16:55:52 -- STEP: 568/4076 -- GLOBAL_STEP: 29100[0m | > loss_text_ce: 0.039239551872015 (0.035610465470596514) | > loss_mel_ce: 3.228222370147705 (3.0622854106862767) | > loss: 0.8168655037879944 (0.7744739695753852) | > current_lr: 5e-06 | > step_time: 0.233 (0.2309102907147206) | > loader_time: 0.0085 (0.009258075079447786) [1m --> TIME: 2024-12-19 16:56:13 -- STEP: 618/4076 -- GLOBAL_STEP: 29150[0m | > loss_text_ce: 0.034204211086034775 (0.035603505815454635) | > loss_mel_ce: 2.898958921432495 (3.062467304248255) | > loss: 0.7332907915115356 (0.7745177028248614) | > current_lr: 5e-06 | > step_time: 0.2094 (0.2311062666204755) | > loader_time: 0.01 (0.009232552306165976) [1m --> TIME: 2024-12-19 16:56:34 -- STEP: 668/4076 -- GLOBAL_STEP: 29200[0m | > loss_text_ce: 0.035714291036129 (0.03556455601240969) | > loss_mel_ce: 3.0602872371673584 (3.062957085892112) | > loss: 0.7740004062652588 (0.7746304109424892) | > current_lr: 5e-06 | > step_time: 0.2366 (0.23097309880627842) | > loader_time: 0.0084 (0.009214264903953691) [1m --> TIME: 2024-12-19 16:56:55 -- STEP: 718/4076 -- GLOBAL_STEP: 29250[0m | > loss_text_ce: 0.03130364045500755 (0.03558305756533956) | > loss_mel_ce: 3.345940589904785 (3.062655697958051) | > loss: 0.8443110585212708 (0.7745596895974994) | > current_lr: 5e-06 | > step_time: 0.3619 (0.23116328955360774) | > loader_time: 0.0089 (0.009188996716130082) [1m --> TIME: 2024-12-19 16:57:16 -- STEP: 768/4076 -- GLOBAL_STEP: 29300[0m | > loss_text_ce: 0.03189175948500633 (0.03557250172161727) | > loss_mel_ce: 2.984057903289795 (3.061364104039967) | > loss: 0.7539874315261841 (0.7742341519333426) | > current_lr: 5e-06 | > step_time: 0.1925 (0.23079937137663364) | > loader_time: 0.0085 (0.009173120992879076) [1m --> TIME: 2024-12-19 16:57:37 -- STEP: 818/4076 -- GLOBAL_STEP: 29350[0m | > loss_text_ce: 0.03820700943470001 (0.03555850442590003) | > loss_mel_ce: 3.049903392791748 (3.061761263821702) | > loss: 0.7720276117324829 (0.774329942421109) | > current_lr: 5e-06 | > step_time: 0.2091 (0.23043637316500937) | > loader_time: 0.0098 (0.00918773858646309) [1m --> TIME: 2024-12-19 16:57:58 -- STEP: 868/4076 -- GLOBAL_STEP: 29400[0m | > loss_text_ce: 0.03579292073845863 (0.035554959099688255) | > loss_mel_ce: 3.0319137573242188 (3.06332038130079) | > loss: 0.766926646232605 (0.7747188356065533) | > current_lr: 5e-06 | > step_time: 0.1556 (0.23063691997308336) | > loader_time: 0.0087 (0.009185133991153569) [1m --> TIME: 2024-12-19 16:58:19 -- STEP: 918/4076 -- GLOBAL_STEP: 29450[0m | > loss_text_ce: 0.03642255812883377 (0.035530199992728106) | > loss_mel_ce: 2.8083596229553223 (3.063187736311769) | > loss: 0.7111955285072327 (0.7746794843855509) | > current_lr: 5e-06 | > step_time: 0.1792 (0.23022277578549188) | > loader_time: 0.0082 (0.009166915951730907) [1m --> TIME: 2024-12-19 16:58:39 -- STEP: 968/4076 -- GLOBAL_STEP: 29500[0m | > loss_text_ce: 0.03287992253899574 (0.03553910439926359) | > loss_mel_ce: 3.1434648036956787 (3.061601646921851) | > loss: 0.7940861582756042 (0.7742851879601639) | > current_lr: 5e-06 | > step_time: 0.1739 (0.22994447468726104) | > loader_time: 0.0098 (0.009169663772110119) [1m --> TIME: 2024-12-19 16:59:00 -- STEP: 1018/4076 -- GLOBAL_STEP: 29550[0m | > loss_text_ce: 0.03179749846458435 (0.035533408398331946) | > loss_mel_ce: 3.062852382659912 (3.0616158862947715) | > loss: 0.7736624479293823 (0.7742873235863649) | > current_lr: 5e-06 | > step_time: 0.2485 (0.2297030488259666) | > loader_time: 0.01 (0.0091708104596391) [1m --> TIME: 2024-12-19 16:59:21 -- STEP: 1068/4076 -- GLOBAL_STEP: 29600[0m | > loss_text_ce: 0.038625918328762054 (0.03554613807688601) | > loss_mel_ce: 3.4229063987731934 (3.063448350081283) | > loss: 0.8653830885887146 (0.7747486220316943) | > current_lr: 5e-06 | > step_time: 0.2089 (0.22964822286077208) | > loader_time: 0.0086 (0.0091849235113194) [1m --> TIME: 2024-12-19 16:59:42 -- STEP: 1118/4076 -- GLOBAL_STEP: 29650[0m | > loss_text_ce: 0.0325135663151741 (0.035516798582880975) | > loss_mel_ce: 3.053988218307495 (3.063575936131485) | > loss: 0.7716254591941833 (0.7747731835769627) | > current_lr: 5e-06 | > step_time: 0.2337 (0.22983292944742656) | > loader_time: 0.0105 (0.00919735346369326) [1m --> TIME: 2024-12-19 17:00:03 -- STEP: 1168/4076 -- GLOBAL_STEP: 29700[0m | > loss_text_ce: 0.03432580828666687 (0.03551486304048958) | > loss_mel_ce: 2.9928126335144043 (3.063391389503871) | > loss: 0.7567846179008484 (0.7747265631205413) | > current_lr: 5e-06 | > step_time: 0.1913 (0.22948655792295117) | > loader_time: 0.0089 (0.009214898903075969) [1m --> TIME: 2024-12-19 17:00:24 -- STEP: 1218/4076 -- GLOBAL_STEP: 29750[0m | > loss_text_ce: 0.03564108908176422 (0.0355219624597084) | > loss_mel_ce: 2.8766801357269287 (3.062997183384762) | > loss: 0.7280803322792053 (0.7746297866350715) | > current_lr: 5e-06 | > step_time: 0.2318 (0.2295420835366586) | > loader_time: 0.0084 (0.009213445809087149) [1m --> TIME: 2024-12-19 17:00:44 -- STEP: 1268/4076 -- GLOBAL_STEP: 29800[0m | > loss_text_ce: 0.03656609356403351 (0.035542934602684706) | > loss_mel_ce: 3.0402071475982666 (3.0628689959974307) | > loss: 0.7691932916641235 (0.7746029828061068) | > current_lr: 5e-06 | > step_time: 0.191 (0.22919387020147186) | > loader_time: 0.01 (0.009227715832201847) [1m --> TIME: 2024-12-19 17:01:06 -- STEP: 1318/4076 -- GLOBAL_STEP: 29850[0m | > loss_text_ce: 0.03934604674577713 (0.03555613688485802) | > loss_mel_ce: 3.094663619995117 (3.0649141896295604) | > loss: 0.7835023999214172 (0.775117581820452) | > current_lr: 5e-06 | > step_time: 0.2123 (0.22953962718952045) | > loader_time: 0.0093 (0.00923970689903804) [1m --> TIME: 2024-12-19 17:01:26 -- STEP: 1368/4076 -- GLOBAL_STEP: 29900[0m | > loss_text_ce: 0.030705882236361504 (0.035561716983269126) | > loss_mel_ce: 3.0925111770629883 (3.0639841092957365) | > loss: 0.7808042764663696 (0.7748864568029232) | > current_lr: 5e-06 | > step_time: 0.1555 (0.22942337585471526) | > loader_time: 0.0089 (0.009248402557874985) [1m --> TIME: 2024-12-19 17:01:47 -- STEP: 1418/4076 -- GLOBAL_STEP: 29950[0m | > loss_text_ce: 0.03249130770564079 (0.03555794210634253) | > loss_mel_ce: 3.009477138519287 (3.062510715216942) | > loss: 0.7604920864105225 (0.7745171644960036) | > current_lr: 5e-06 | > step_time: 0.2314 (0.22946853176999327) | > loader_time: 0.0123 (0.009258298846999411) [1m --> TIME: 2024-12-19 17:02:09 -- STEP: 1468/4076 -- GLOBAL_STEP: 30000[0m | > loss_text_ce: 0.03353715315461159 (0.035542434012918275) | > loss_mel_ce: 3.180691719055176 (3.0626519414969273) | > loss: 0.8035572171211243 (0.7745485939593018) | > current_lr: 5e-06 | > step_time: 0.2336 (0.22955205330082115) | > loader_time: 0.0107 (0.00926734722277773) [1m --> TIME: 2024-12-19 17:02:29 -- STEP: 1518/4076 -- GLOBAL_STEP: 30050[0m | > loss_text_ce: 0.03730197250843048 (0.03552900047602227) | > loss_mel_ce: 3.1681690216064453 (3.0636230350640115) | > loss: 0.8013677597045898 (0.774788009005259) | > current_lr: 5e-06 | > step_time: 0.1777 (0.22922104055231268) | > loader_time: 0.0085 (0.009273892963042527) [1m --> TIME: 2024-12-19 17:02:50 -- STEP: 1568/4076 -- GLOBAL_STEP: 30100[0m | > loss_text_ce: 0.03651067614555359 (0.0355493441195588) | > loss_mel_ce: 2.8126680850982666 (3.0639533931199363) | > loss: 0.7122946977615356 (0.7748756844322293) | > current_lr: 5e-06 | > step_time: 0.1894 (0.2291268029687356) | > loader_time: 0.0085 (0.00927555758734138) [1m --> TIME: 2024-12-19 17:03:10 -- STEP: 1618/4076 -- GLOBAL_STEP: 30150[0m | > loss_text_ce: 0.03925441950559616 (0.03555447233170818) | > loss_mel_ce: 2.849360704421997 (3.0616599025184033) | > loss: 0.7221537828445435 (0.7743035939479495) | > current_lr: 5e-06 | > step_time: 0.2108 (0.2289444916917308) | > loader_time: 0.0085 (0.009280558274909206) [1m --> TIME: 2024-12-19 17:03:31 -- STEP: 1668/4076 -- GLOBAL_STEP: 30200[0m | > loss_text_ce: 0.031224416568875313 (0.03555614402522139) | > loss_mel_ce: 3.274742841720581 (3.061569994587976) | > loss: 0.8264918327331543 (0.7742815348956226) | > current_lr: 5e-06 | > step_time: 0.191 (0.22864402383923244) | > loader_time: 0.0089 (0.009270760795767063) [1m --> TIME: 2024-12-19 17:03:52 -- STEP: 1718/4076 -- GLOBAL_STEP: 30250[0m | > loss_text_ce: 0.032567478716373444 (0.035563589616898754) | > loss_mel_ce: 3.0384669303894043 (3.062071783439936) | > loss: 0.7677586078643799 (0.7744088434506907) | > current_lr: 5e-06 | > step_time: 0.207 (0.22874014619065663) | > loader_time: 0.0085 (0.009266269082101583) [1m --> TIME: 2024-12-19 17:04:12 -- STEP: 1768/4076 -- GLOBAL_STEP: 30300[0m | > loss_text_ce: 0.034783389419317245 (0.03557667857861963) | > loss_mel_ce: 2.9877421855926514 (3.06180862563228) | > loss: 0.7556313872337341 (0.7743463262713335) | > current_lr: 5e-06 | > step_time: 0.173 (0.22858077004484462) | > loader_time: 0.0099 (0.009255301628716923) [1m --> TIME: 2024-12-19 17:04:33 -- STEP: 1818/4076 -- GLOBAL_STEP: 30350[0m | > loss_text_ce: 0.03804430738091469 (0.03556150595715926) | > loss_mel_ce: 3.2161073684692383 (3.0623313955741334) | > loss: 0.8135378956794739 (0.7744732255523881) | > current_lr: 5e-06 | > step_time: 0.2753 (0.22857985808642128) | > loader_time: 0.0103 (0.009252011710398784) [1m --> TIME: 2024-12-19 17:04:54 -- STEP: 1868/4076 -- GLOBAL_STEP: 30400[0m | > loss_text_ce: 0.03572720289230347 (0.03556182766851731) | > loss_mel_ce: 3.337959051132202 (3.061983172592308) | > loss: 0.8434215784072876 (0.7743862502643843) | > current_lr: 5e-06 | > step_time: 0.173 (0.2284698052375679) | > loader_time: 0.0081 (0.009250350631534152) [1m --> TIME: 2024-12-19 17:05:15 -- STEP: 1918/4076 -- GLOBAL_STEP: 30450[0m | > loss_text_ce: 0.03178572282195091 (0.03557251821776096) | > loss_mel_ce: 3.087050676345825 (3.062269754404816) | > loss: 0.7797091007232666 (0.7744605682937894) | > current_lr: 5e-06 | > step_time: 0.2296 (0.22842355341309675) | > loader_time: 0.0097 (0.009254870996485157) [1m --> TIME: 2024-12-19 17:05:36 -- STEP: 1968/4076 -- GLOBAL_STEP: 30500[0m | > loss_text_ce: 0.03783433884382248 (0.03557464864949233) | > loss_mel_ce: 3.2175323963165283 (3.062982506141428) | > loss: 0.813841700553894 (0.7746392887661129) | > current_lr: 5e-06 | > step_time: 0.2315 (0.2285870506269176) | > loader_time: 0.0104 (0.009257038070903554) [1m --> TIME: 2024-12-19 17:05:57 -- STEP: 2018/4076 -- GLOBAL_STEP: 30550[0m | > loss_text_ce: 0.033692892640829086 (0.035572919273717606) | > loss_mel_ce: 3.3126282691955566 (3.0635833689431586) | > loss: 0.8365802764892578 (0.7747890722113631) | > current_lr: 5e-06 | > step_time: 0.2323 (0.22879386668871604) | > loader_time: 0.0109 (0.009254829143981624) [1m --> TIME: 2024-12-19 17:06:19 -- STEP: 2068/4076 -- GLOBAL_STEP: 30600[0m | > loss_text_ce: 0.035303257405757904 (0.035587854581620226) | > loss_mel_ce: 3.2273900508880615 (3.0643359101026135) | > loss: 0.8156733512878418 (0.7749809412073122) | > current_lr: 5e-06 | > step_time: 0.1569 (0.22890542692557064) | > loader_time: 0.0087 (0.00925657516291339) [1m --> TIME: 2024-12-19 17:06:39 -- STEP: 2118/4076 -- GLOBAL_STEP: 30650[0m | > loss_text_ce: 0.03462950512766838 (0.035588602563568564) | > loss_mel_ce: 2.80560302734375 (3.062909910654096) | > loss: 0.7100581526756287 (0.7746246282057002) | > current_lr: 5e-06 | > step_time: 0.2121 (0.22884728032761417) | > loader_time: 0.0092 (0.009249395756816041) [1m --> TIME: 2024-12-19 17:07:00 -- STEP: 2168/4076 -- GLOBAL_STEP: 30700[0m | > loss_text_ce: 0.03386528789997101 (0.035599696706822385) | > loss_mel_ce: 3.199718713760376 (3.063033169376011) | > loss: 0.8083959817886353 (0.7746582164315722) | > current_lr: 5e-06 | > step_time: 0.192 (0.22870981451330152) | > loader_time: 0.0093 (0.009241256766653577) [1m --> TIME: 2024-12-19 17:07:20 -- STEP: 2218/4076 -- GLOBAL_STEP: 30750[0m | > loss_text_ce: 0.036340873688459396 (0.0355868535603391) | > loss_mel_ce: 2.7563557624816895 (3.0610786793573745) | > loss: 0.6981741786003113 (0.7741663832187224) | > current_lr: 5e-06 | > step_time: 0.2756 (0.22854270023521378) | > loader_time: 0.0096 (0.009228287461214596) [1m --> TIME: 2024-12-19 17:07:41 -- STEP: 2268/4076 -- GLOBAL_STEP: 30800[0m | > loss_text_ce: 0.035193782299757004 (0.03556579097898474) | > loss_mel_ce: 3.136554002761841 (3.061005498786869) | > loss: 0.7929369211196899 (0.7741428223394217) | > current_lr: 5e-06 | > step_time: 0.2675 (0.2284763191532626) | > loader_time: 0.0093 (0.009220976047414935) [1m --> TIME: 2024-12-19 17:08:02 -- STEP: 2318/4076 -- GLOBAL_STEP: 30850[0m | > loss_text_ce: 0.034055016934871674 (0.03557045284154038) | > loss_mel_ce: 2.9257495403289795 (3.060967941650342) | > loss: 0.7399511337280273 (0.7741345985373927) | > current_lr: 5e-06 | > step_time: 0.2323 (0.22867834784019805) | > loader_time: 0.0086 (0.009212344447737405) [1m --> TIME: 2024-12-19 17:08:24 -- STEP: 2368/4076 -- GLOBAL_STEP: 30900[0m | > loss_text_ce: 0.03735338896512985 (0.03558388035083693) | > loss_mel_ce: 3.0177366733551025 (3.060223571251372) | > loss: 0.7637724876403809 (0.7739518628631895) | > current_lr: 5e-06 | > step_time: 0.2739 (0.22871841530542117) | > loader_time: 0.0108 (0.009216672564680495) [1m --> TIME: 2024-12-19 17:08:45 -- STEP: 2418/4076 -- GLOBAL_STEP: 30950[0m | > loss_text_ce: 0.03077876940369606 (0.035580484137999004) | > loss_mel_ce: 3.4022510051727295 (3.0605469585155225) | > loss: 0.8582574725151062 (0.7740318607061334) | > current_lr: 5e-06 | > step_time: 0.186 (0.22882546098691753) | > loader_time: 0.0082 (0.009222860371800258) [1m --> TIME: 2024-12-19 17:09:06 -- STEP: 2468/4076 -- GLOBAL_STEP: 31000[0m | > loss_text_ce: 0.037569157779216766 (0.03557383838204764) | > loss_mel_ce: 2.964667320251465 (3.060851470191435) | > loss: 0.7505590915679932 (0.7741063271726156) | > current_lr: 5e-06 | > step_time: 0.2362 (0.2288552994086638) | > loader_time: 0.0088 (0.009218143489411027) [1m --> TIME: 2024-12-19 17:09:27 -- STEP: 2518/4076 -- GLOBAL_STEP: 31050[0m | > loss_text_ce: 0.03460792079567909 (0.035566117780947364) | > loss_mel_ce: 2.901860475540161 (3.060148491946544) | > loss: 0.7341170907020569 (0.7739286524516609) | > current_lr: 5e-06 | > step_time: 0.232 (0.22884918082981853) | > loader_time: 0.0091 (0.009218629434811283) [1m --> TIME: 2024-12-19 17:09:48 -- STEP: 2568/4076 -- GLOBAL_STEP: 31100[0m | > loss_text_ce: 0.034760139882564545 (0.03556751110155799) | > loss_mel_ce: 2.9137351512908936 (3.0592508724545366) | > loss: 0.7371238470077515 (0.7737045959102399) | > current_lr: 5e-06 | > step_time: 0.269 (0.22883501834587144) | > loader_time: 0.0105 (0.009224395607119386) [1m --> TIME: 2024-12-19 17:10:08 -- STEP: 2618/4076 -- GLOBAL_STEP: 31150[0m | > loss_text_ce: 0.03359303995966911 (0.035553660316320186) | > loss_mel_ce: 2.987497329711914 (3.0577578042826623) | > loss: 0.7552725672721863 (0.7733278661579276) | > current_lr: 5e-06 | > step_time: 0.1863 (0.22869866316877677) | > loader_time: 0.008 (0.00921703623486801) [1m --> TIME: 2024-12-19 17:10:29 -- STEP: 2668/4076 -- GLOBAL_STEP: 31200[0m | > loss_text_ce: 0.03217965364456177 (0.03555146535660238) | > loss_mel_ce: 3.36833119392395 (3.058746578096449) | > loss: 0.8501276969909668 (0.7735745108824613) | > current_lr: 5e-06 | > step_time: 0.1916 (0.2288095454226012) | > loader_time: 0.0087 (0.009207458689116259) [1m --> TIME: 2024-12-19 17:10:50 -- STEP: 2718/4076 -- GLOBAL_STEP: 31250[0m | > loss_text_ce: 0.03151800483465195 (0.0355484144387985) | > loss_mel_ce: 2.799966335296631 (3.0578978481847106) | > loss: 0.7078710794448853 (0.7733615656781488) | > current_lr: 5e-06 | > step_time: 0.2127 (0.2288167966122659) | > loader_time: 0.0082 (0.009203979774289798) [1m --> TIME: 2024-12-19 17:11:11 -- STEP: 2768/4076 -- GLOBAL_STEP: 31300[0m | > loss_text_ce: 0.03768279030919075 (0.03554427680206845) | > loss_mel_ce: 3.284214496612549 (3.0576071640142817) | > loss: 0.830474317073822 (0.7732878601921425) | > current_lr: 5e-06 | > step_time: 0.1923 (0.2288391679697643) | > loader_time: 0.0083 (0.009201549984127075) [1m --> TIME: 2024-12-19 17:11:33 -- STEP: 2818/4076 -- GLOBAL_STEP: 31350[0m | > loss_text_ce: 0.033934030681848526 (0.03553752006858462) | > loss_mel_ce: 3.0675835609436035 (3.057507543127278) | > loss: 0.7753794193267822 (0.7732612657961558) | > current_lr: 5e-06 | > step_time: 0.277 (0.228881751247295) | > loader_time: 0.0086 (0.009197481901955285) [1m --> TIME: 2024-12-19 17:11:53 -- STEP: 2868/4076 -- GLOBAL_STEP: 31400[0m | > loss_text_ce: 0.03222336620092392 (0.03553466876576469) | > loss_mel_ce: 3.1624672412872314 (3.056806887875375) | > loss: 0.7986726760864258 (0.7730853891563875) | > current_lr: 5e-06 | > step_time: 0.1513 (0.22880587409896) | > loader_time: 0.0101 (0.009198045215181028) [1m --> TIME: 2024-12-19 17:12:13 -- STEP: 2918/4076 -- GLOBAL_STEP: 31450[0m | > loss_text_ce: 0.03765365481376648 (0.035544179775704246) | > loss_mel_ce: 3.1055235862731934 (3.056444340490821) | > loss: 0.7857943177223206 (0.7729971300379058) | > current_lr: 5e-06 | > step_time: 0.2759 (0.2286541247874437) | > loader_time: 0.009 (0.009195918902867096) [1m --> TIME: 2024-12-19 17:12:35 -- STEP: 2968/4076 -- GLOBAL_STEP: 31500[0m | > loss_text_ce: 0.040964748710393906 (0.03553939704102965) | > loss_mel_ce: 2.9171693325042725 (3.0559879629759776) | > loss: 0.739533543586731 (0.7728818400168027) | > current_lr: 5e-06 | > step_time: 0.1731 (0.22882184940528355) | > loader_time: 0.0082 (0.009193682445670042) [1m --> TIME: 2024-12-19 17:12:55 -- STEP: 3018/4076 -- GLOBAL_STEP: 31550[0m | > loss_text_ce: 0.03939111903309822 (0.03554722456837956) | > loss_mel_ce: 2.7176623344421387 (3.0549071386525615) | > loss: 0.6892633438110352 (0.7726135908214355) | > current_lr: 5e-06 | > step_time: 0.2646 (0.22862424654546679) | > loader_time: 0.0086 (0.009187389557688627) [1m --> TIME: 2024-12-19 17:13:16 -- STEP: 3068/4076 -- GLOBAL_STEP: 31600[0m | > loss_text_ce: 0.03733986243605614 (0.035546351528958395) | > loss_mel_ce: 3.150463581085205 (3.054418988224736) | > loss: 0.7969508767127991 (0.7724913349557259) | > current_lr: 5e-06 | > step_time: 0.1595 (0.22852803512222608) | > loader_time: 0.0079 (0.009182029628380616) [1m --> TIME: 2024-12-19 17:13:36 -- STEP: 3118/4076 -- GLOBAL_STEP: 31650[0m | > loss_text_ce: 0.03725471347570419 (0.03554505303339586) | > loss_mel_ce: 3.007819175720215 (3.053729743358032) | > loss: 0.7612684965133667 (0.772318699177906) | > current_lr: 5e-06 | > step_time: 0.1826 (0.22850944439213272) | > loader_time: 0.0078 (0.00916970823727177) [1m --> TIME: 2024-12-19 17:13:57 -- STEP: 3168/4076 -- GLOBAL_STEP: 31700[0m | > loss_text_ce: 0.03706405311822891 (0.03554367024164565) | > loss_mel_ce: 2.933781623840332 (3.053774474922455) | > loss: 0.7427114248275757 (0.7723295363701046) | > current_lr: 5e-06 | > step_time: 0.1562 (0.22848999605636405) | > loader_time: 0.0106 (0.00916959044307166) [1m --> TIME: 2024-12-19 17:14:18 -- STEP: 3218/4076 -- GLOBAL_STEP: 31750[0m | > loss_text_ce: 0.04091611132025719 (0.035542285766723365) | > loss_mel_ce: 2.8877804279327393 (3.054514109014057) | > loss: 0.7321741580963135 (0.7725140987678366) | > current_lr: 5e-06 | > step_time: 0.213 (0.22853901684098987) | > loader_time: 0.0096 (0.009173851564406128) [1m --> TIME: 2024-12-19 17:14:39 -- STEP: 3268/4076 -- GLOBAL_STEP: 31800[0m | > loss_text_ce: 0.040310584008693695 (0.03555153478153562) | > loss_mel_ce: 3.2268970012664795 (3.0546562806629054) | > loss: 0.816801905632019 (0.7725519538837659) | > current_lr: 5e-06 | > step_time: 0.2184 (0.22851324438902856) | > loader_time: 0.0086 (0.009172109305639706) [1m --> TIME: 2024-12-19 17:15:00 -- STEP: 3318/4076 -- GLOBAL_STEP: 31850[0m | > loss_text_ce: 0.03266198933124542 (0.03555371918260251) | > loss_mel_ce: 3.191255807876587 (3.0555495445522793) | > loss: 0.8059794306755066 (0.7727758159401757) | > current_lr: 5e-06 | > step_time: 0.209 (0.228412069779983) | > loader_time: 0.0084 (0.009169210792665632) [1m --> TIME: 2024-12-19 17:15:20 -- STEP: 3368/4076 -- GLOBAL_STEP: 31900[0m | > loss_text_ce: 0.03582451865077019 (0.035548745806609806) | > loss_mel_ce: 2.9780662059783936 (3.0548266699789823) | > loss: 0.7534726858139038 (0.772593854017325) | > current_lr: 5e-06 | > step_time: 0.1937 (0.22836928915524426) | > loader_time: 0.0086 (0.009164373268707305) [1m --> TIME: 2024-12-19 17:15:41 -- STEP: 3418/4076 -- GLOBAL_STEP: 31950[0m | > loss_text_ce: 0.03708158805966377 (0.03554413379853905) | > loss_mel_ce: 3.2537550926208496 (3.0543840769231494) | > loss: 0.8227091431617737 (0.7724820527331456) | > current_lr: 5e-06 | > step_time: 0.2271 (0.2282634916327863) | > loader_time: 0.0105 (0.009160403622180234) [1m --> TIME: 2024-12-19 17:16:02 -- STEP: 3468/4076 -- GLOBAL_STEP: 32000[0m | > loss_text_ce: 0.038495369255542755 (0.035539370271857364) | > loss_mel_ce: 3.0060439109802246 (3.053831501678062) | > loss: 0.7611348032951355 (0.7723427180291041) | > current_lr: 5e-06 | > step_time: 0.1897 (0.22834002854777316) | > loader_time: 0.0086 (0.00916110114930269) [1m --> TIME: 2024-12-19 17:16:22 -- STEP: 3518/4076 -- GLOBAL_STEP: 32050[0m | > loss_text_ce: 0.03695579990744591 (0.03553085231008835) | > loss_mel_ce: 3.4435136318206787 (3.0528458020035147) | > loss: 0.8701173663139343 (0.772094163639156) | > current_lr: 5e-06 | > step_time: 0.3316 (0.22825252969133508) | > loader_time: 0.0101 (0.009157493007393129) [1m --> TIME: 2024-12-19 17:16:43 -- STEP: 3568/4076 -- GLOBAL_STEP: 32100[0m | > loss_text_ce: 0.036101434379816055 (0.035531644551751645) | > loss_mel_ce: 3.0563433170318604 (3.0526328175725426) | > loss: 0.7731111645698547 (0.7720411155757075) | > current_lr: 5e-06 | > step_time: 0.1576 (0.22818961933322016) | > loader_time: 0.0092 (0.009157649872014305) [1m --> TIME: 2024-12-19 17:17:04 -- STEP: 3618/4076 -- GLOBAL_STEP: 32150[0m | > loss_text_ce: 0.02942121960222721 (0.03552892128229057) | > loss_mel_ce: 3.218143939971924 (3.0522667402190486) | > loss: 0.8118913173675537 (0.7719489153970858) | > current_lr: 5e-06 | > step_time: 0.2134 (0.22827562190604644) | > loader_time: 0.0097 (0.009157853669524487) [1m --> TIME: 2024-12-19 17:17:25 -- STEP: 3668/4076 -- GLOBAL_STEP: 32200[0m | > loss_text_ce: 0.03353916481137276 (0.035523460933180956) | > loss_mel_ce: 3.0139119625091553 (3.0522257066865004) | > loss: 0.7618627548217773 (0.7719372919595093) | > current_lr: 5e-06 | > step_time: 0.2648 (0.22822541986452974) | > loader_time: 0.0091 (0.009159947624643306) [1m --> TIME: 2024-12-19 17:17:46 -- STEP: 3718/4076 -- GLOBAL_STEP: 32250[0m | > loss_text_ce: 0.036871302872896194 (0.035519356277759755) | > loss_mel_ce: 3.2441487312316895 (3.051366567291067) | > loss: 0.8202549815177917 (0.7717214808982179) | > current_lr: 5e-06 | > step_time: 0.2596 (0.2282969055796516) | > loader_time: 0.0084 (0.009153909039407566) [1m --> TIME: 2024-12-19 17:18:07 -- STEP: 3768/4076 -- GLOBAL_STEP: 32300[0m | > loss_text_ce: 0.03843207657337189 (0.0355195763067099) | > loss_mel_ce: 3.1971423625946045 (3.050837474011059) | > loss: 0.808893620967865 (0.7715892625652296) | > current_lr: 5e-06 | > step_time: 0.156 (0.22825074284446215) | > loader_time: 0.0081 (0.009156486727376366) [1m --> TIME: 2024-12-19 17:18:28 -- STEP: 3818/4076 -- GLOBAL_STEP: 32350[0m | > loss_text_ce: 0.032145313918590546 (0.03551189591420248) | > loss_mel_ce: 2.7546350955963135 (3.0496834666395514) | > loss: 0.69669508934021 (0.7712988405873348) | > current_lr: 5e-06 | > step_time: 0.2318 (0.22823576386403263) | > loader_time: 0.0086 (0.009153060540908132) [1m --> TIME: 2024-12-19 17:18:49 -- STEP: 3868/4076 -- GLOBAL_STEP: 32400[0m | > loss_text_ce: 0.035469092428684235 (0.035513950515184016) | > loss_mel_ce: 3.0867204666137695 (3.0495055904679362) | > loss: 0.7805473804473877 (0.7712548852059311) | > current_lr: 5e-06 | > step_time: 0.1896 (0.22821089924306387) | > loader_time: 0.0087 (0.009146954860411081) [1m --> TIME: 2024-12-19 17:19:09 -- STEP: 3918/4076 -- GLOBAL_STEP: 32450[0m | > loss_text_ce: 0.033738620579242706 (0.035510931768268715) | > loss_mel_ce: 2.9553675651550293 (3.0497404769587844) | > loss: 0.7472765445709229 (0.7713128521680949) | > current_lr: 5e-06 | > step_time: 0.207 (0.22819489669653759) | > loader_time: 0.0092 (0.009145640122032424) [1m --> TIME: 2024-12-19 17:19:30 -- STEP: 3968/4076 -- GLOBAL_STEP: 32500[0m | > loss_text_ce: 0.036693423986434937 (0.035508717807862265) | > loss_mel_ce: 3.166788101196289 (3.049542006585867) | > loss: 0.8008703589439392 (0.7712626810966716) | > current_lr: 5e-06 | > step_time: 0.1894 (0.22815314488064858) | > loader_time: 0.0101 (0.009143156810633565) [1m --> TIME: 2024-12-19 17:19:51 -- STEP: 4018/4076 -- GLOBAL_STEP: 32550[0m | > loss_text_ce: 0.03638947755098343 (0.03550531360890479) | > loss_mel_ce: 2.6791832447052 (3.049339332943713) | > loss: 0.6788932085037231 (0.7712111616087169) | > current_lr: 5e-06 | > step_time: 0.2303 (0.22813344304391908) | > loader_time: 0.0101 (0.009142347329169584) [1m --> TIME: 2024-12-19 17:20:12 -- STEP: 4068/4076 -- GLOBAL_STEP: 32600[0m | > loss_text_ce: 0.03700932487845421 (0.03550240610824162) | > loss_mel_ce: 2.9976913928985596 (3.048558742892191) | > loss: 0.7586751580238342 (0.771015287178221) | > current_lr: 5e-06 | > step_time: 0.1739 (0.22809610248432516) | > loader_time: 0.0078 (0.009144640250191896) [1m > EVALUATION [0m [1m--> EVAL PERFORMANCE[0m | > avg_loader_time:[92m 0.4690309946353619 [0m(-0.0023535031538742812) | > avg_loss_text_ce:[92m 0.03420695347281603 [0m(-0.00023809920709867705) | > avg_loss_mel_ce:[92m 3.16597227866833 [0m(-0.009506546534024718) | > avg_loss:[92m 3.200179237585801 [0m(-0.009744644165039062) > BEST MODEL : /tmp/checkpoints/GPT_XTTS_FT-December-19-2024_01+24PM-9b9aedb/best_model_32608.pth