compress loss = 0 about nncf

[lzrvch]

Hi @YoungSharp,

The compression algorithm loss is a generic entity in our API and it ensures that any compression algorithm (or a combination thereof) works correctly, including the cases when it requires adding a penalty term to your main loss function. Some algorithms do not have such a requirement, such as quantization and magnitude sparsity, and for them, this loss term is 0 by default.

I suppose you are using one of those algorithms to compress your model? In that case, this behavior is expected. Do you have any other problems with your model fine-tuning procedure?

[YoungSharp]

Hi @vanyalzr

thanks for replay, I was using quantization algorithm to compress my model.
If compression_loss = 0 is expected. I have another problem. By using mmdetection onnx exporter to export my compressed model to onnx model. i find onnx model weight does not looks like been quantized.
quantization config:
input_size = 448 ENABLE_COMPRESSION = True nncf_config = dict(compression=[dict(algorithm='quantization', initializer=dict(range=dict(num_init_steps=10)))], log_dir=work_dir)
Base model is efficientnet-b3, the first conv layer weights is below.
[ [ [ [ 0.3603667616844177, -0.5374307632446289, 0.001952380989678204 ], [ 0.6092172265052795, -0.5033608675003052, -0.051245443522930145 ], [ 0.015762008726596832, -0.0003829538472928107, 0.06971737742424011 ] ], [ [ 1.3720226287841797, -1.4568500518798828, -0.012406611815094948 ], [ 1.2627625465393066, -1.3461413383483887, -0.05189552903175354 ], [ 0.15398229658603668, 0.03449089452624321, 0.08602666854858398 ] ], [ [ 0.15650884807109833, -0.2866743206977844, 0.022628290578722954 ], [ 0.3569646179676056, -0.3498978614807129, -0.016743917018175125 ], [ 0.00387936201877892, -0.011319339275360107, 0.04645385965704918 ] ] ], [ [ [ 0.02379395067691803, 0.06416888535022736, 0.03441089391708374 ], [ -1.3542351722717285, -0.014414608478546143, 0.002027014037594199 ], [ -0.06681133806705475, 0.042854949831962585, 0.017452307045459747 ] ], [ [ 0.11277637630701065, -0.04625503346323967, 0.03549974411725998 ], [ -0.6202108263969421, 0.038116853684186935, 0.01379059161990881 ], [ 0.04088037833571434, 0.029540065675973892, 0.032375186681747437 ] ], [ [ 0.025897245854139328, 0.08049408346414566, 0.025638259947299957 ], [ -0.14927999675273895, 0.1072687953710556, 0.012837743386626244 ], [ -0.008325441740453243, 0.055185429751873016, 0.021701721474528313 ] ] ], [ [ [ 0.8385065793991089, 1.1928296089172363, 0.28208914399147034 ], [ 1.0477042198181152, 1.5227644443511963, 0.28695428371429443 ], [ 0.3125009536743164, 0.3149097263813019, -0.02771533466875553 ] ], [ [ -0.5980737209320068, -0.9167705178260803, -0.3587746322154999 ], [ -0.761725664138794, -1.0621089935302734, -0.38351115584373474 ], [ -0.2125093787908554, -0.3627846837043762, -0.06995101273059845 ] ], [ [ -0.25617456436157227, -0.3751916289329529, -0.10125856101512909 ], [ -0.33208248019218445, -0.5034953355789185, -0.06561792641878128 ], [ -0.0245540589094162, -0.04001801460981369, -0.014760272577404976 ] ] ], [ [ [ 0.016360608860850334, -0.3531891703605652, 0.03414047136902809 ], [ 0.09571370482444763, -0.08611265569925308, 0.0549452118575573 ], [ 0.06656917184591293, 0.013940572738647461, 0.04504940286278725 ] ], [ [ -0.27257680892944336, -1.6714335680007935, -0.09305314719676971 ], [ -0.09260738641023636, -0.17229221761226654, -0.039645005017519 ], [ -0.03724232316017151, -0.03694054111838341, -0.06745678186416626 ] ], [ [ 0.034323159605264664, -0.22521057724952698, 0.009854068048298359 ], [ 0.0744839608669281, 0.015600468963384628, 0.013111146166920662 ], [ 0.0033704806119203568, 0.050526298582553864, 0.016064150258898735 ] ] ], [ [ [ 0.7831529974937439, 0.08436092734336853, -0.05339992418885231 ], [ -0.08739947527647018, -0.06245709955692291, -0.03166312724351883 ], [ -0.016141800209879875, -0.04678699001669884, -0.03231404349207878 ] ], [ [ 1.3972235918045044, -0.03711560368537903, 0.03354016691446304 ], [ -0.015348229557275772, 0.07899950444698334, -0.00017750493134371936 ], [ 0.06583419442176819, 0.016041243448853493, 0.05606088414788246 ] ], [ [ 0.23624137043952942, 0.008777287788689137, 0.019887495785951614 ], [ -0.05615142360329628, -0.05744911730289459, -0.0127479936927557 ], [ 0.008637518621981144, -0.015697035938501358, -0.017470166087150574 ] ] ], [ [ [ 0.2583921253681183, 0.22606436908245087, -0.012629884295165539 ], [ 0.132790207862854, 0.23452122509479523, -0.05039222165942192 ], [ 0.10378012806177139, -0.00000580014875595225, 0.18373656272888184 ] ], [ [ 0.31156283617019653, 0.3494468629360199, 0.031576622277498245 ], [ 0.23437529802322388, -0.02659810520708561, 0.010435579344630241 ], [ 0.038272518664598465, 0.06256204843521118, -0.045725829899311066 ] ], [ [ -0.6872433423995972, -0.6243017911911011, -0.11017423123121262 ], [ -0.5409526824951172, -0.29402488470077515, -0.025335142388939857 ], [ -0.13306793570518494, -0.05541832372546196, -0.08850867301225662 ] ] ], [ [ [ 0.04774850606918335, -0.008904614485800266, 0.10948716104030609 ], [ -0.048879027366638184, -0.23575983941555023, 0.10958265513181686 ], [ 0.05797131732106209, 0.08702291548252106, -0.012438972480595112 ] ], [ [ -0.0015515829436481, -0.09604127705097198, 0.08865989744663239 ], [ -0.12140224128961563, -0.36449360847473145, 0.121602363884449 ], [ 0.031265731900930405, 0.13444119691848755, 0.00010512386506889015 ] ], [ [ -0.5229217410087585, -0.5190733671188354, -0.24117131531238556 ], [ -0.6028639078140259, -0.7049451470375061, -0.22954948246479034 ], [ -0.20107924938201904, -0.20751909911632538, -0.029963400214910507 ] ] ], [ [ [ 0.8163717985153198, -0.7485648393630981, -0.1100936308503151 ], [ 0.35844388604164124, -0.3826475143432617, 0.0250982865691185 ], [ 0.050180524587631226, -0.04203492030501366, -0.006629049777984619 ] ], [ [ 1.70492684841156, -2.180729627609253, -0.0662025734782219 ], [ 1.4882888793945312, -1.1406952142715454, 0.04071129485964775 ], [ 0.10040322691202164, -0.06366609781980515, 0.031120089814066887 ] ], [ [ 0.3348841071128845, -0.27060186862945557, -0.10768462717533112 ], [ 0.1670825481414795, -0.1599094718694687, -0.0003990217810496688 ], [ 0.040604714304208755, -0.04026969522237778, 0.002382713370025158 ] ] ], [ [ [ 0.4193877577781677, 0.5757598876953125, 0.2693241536617279 ], [ 0.48072150349617004, 0.5980150103569031, 0.27179908752441406 ], [ 0.29319897294044495, 0.2774105966091156, 0.08206933736801147 ] ], [ [ 0.030476650223135948, 0.1904967576265335, -0.12982256710529327 ], [ 0.20038177073001862, 0.2153308242559433, -0.10618191212415695 ], [ -0.12990781664848328, -0.14786118268966675, -0.1167616918683052 ] ], [ [ -0.03919680416584015, 0.20057658851146698, -0.1334412395954132 ], [ 0.11582577973604202, 0.16829605400562286, -0.09570185840129852 ], [ -0.10197532922029495, -0.0907364934682846, 0.015070403926074505 ] ] ], [ [ [ -0.42778220772743225, -0.5743306875228882, -0.1154937893152237 ], [ -0.5544014573097229, -0.6663929224014282, -0.040096547454595566 ], [ -0.04417478293180466, -0.05133568122982979, 0.021676132455468178 ] ], [ [ 0.7522192001342773, 0.8633449673652649, 0.3881658613681793 ], [ 0.7469053864479065, 0.9316321015357971, 0.4451141357421875 ], [ 0.2874215841293335, 0.3667811155319214, 0.13646817207336426 ] ], [ [ -0.16900835931301117, -0.45734903216362, -0.25859203934669495 ], [ -0.36283189058303833, -0.5795014500617981, -0.29318103194236755 ], [ -0.019782477989792824, -0.212410107254982, -0.1644723117351532 ] ] ], [ [ [ 0.019307604059576988, -0.03504905849695206, -0.26730766892433167 ], [ -0.05913460999727249, 0.08116823434829712, 0.35277870297431946 ], [ 0.04194584861397743, 0.05111641809344292, -0.03491010144352913 ] ], [ [ 0.06728450208902359, -0.045907534658908844, -1.0424983501434326 ], [ -0.04568254202604294, 0.04519440606236458, 0.9118595719337463 ], [ 0.10482602566480637, 0.006077687256038189, -0.12225450575351715 ] ], [ [ -0.01152153592556715, -0.011266634799540043, -0.17383696138858795 ], [ -0.0345972441136837, 0.041663769632577896, 0.2710641622543335 ], [ 0.006854290142655373, 0.005612744949758053, -0.003332367865368724 ] ] ], [ [ [ 0.009159397333860397, -0.023808464407920837, 0.014016272500157356 ], [ -0.05235389247536659, -0.6219943165779114, -0.037211500108242035 ], [ -0.014956335537135601, -0.03987602889537811, 0.021333735436201096 ] ], [ [ -0.028101911768317223, 0.038176778703927994, -0.01768294908106327 ], [ 0.019760092720389366, -0.17838062345981598, 0.025721170008182526 ], [ -0.010215799324214458, 0.016402726992964745, 0.0010135842021554708 ] ], [ [ -0.0003243161190766841, 0.01808064430952072, 0.0008097233949229121 ], [ 0.012368757277727127, -0.2519064247608185, 0.001306490390561521 ], [ -0.028058134019374847, -0.00843738205730915, -0.033520009368658066 ] ] ], [ [ [ -0.398679256439209, -0.21350063383579254, 0.32981982827186584 ], [ -0.3684980869293213, -0.086346335709095, 0.19863522052764893 ], [ -0.013920797035098076, 0.1418110430240631, 0.19478055834770203 ] ], [ [ -0.2506774663925171, -0.0494108684360981, 0.18298570811748505 ], [ -0.325312077999115, -0.16771648824214935, 0.22042930126190186 ], [ 0.03484612703323364, 0.13724569976329803, 0.11828869581222534 ] ], [ [ -0.07061946392059326, -0.13229741156101227, 0.15138204395771027 ], [ -0.12568449974060059, -0.06493557244539261, 0.09958207607269287 ], [ -0.0021261044312268496, 0.07625563442707062, 0.06583329290151596 ] ] ], [ [ [ -0.1981441229581833, -0.10210604220628738, -0.004398610442876816 ], [ 0.6559460759162903, 0.29809072613716125, 0.2626305818557739 ], [ -0.0855003148317337, -0.696674108505249, -0.16444475948810577 ] ], [ [ -0.2445569485425949, -0.23681049048900604, -0.06361504644155502 ], [ 0.987004280090332, 0.4560335874557495, 0.39156538248062134 ], [ -0.5248368382453918, -0.5445421934127808, -0.29503196477890015 ] ], [ [ -0.08954138308763504, -0.08140949159860611, -0.01067945547401905 ], [ 0.4982849657535553, 0.2158895581960678, 0.18374638259410858 ], [ 0.060414861887693405, -0.5450943112373352, -0.14695507287979126 ] ] ], [ [ [ -0.8562104105949402, -0.9906357526779175, -0.5031142234802246 ], [ -0.9071540832519531, -0.9781914353370667, -0.4355827867984772 ], [ -0.5464594960212708, -0.500324010848999, -0.22130411863327026 ] ], [ [ 0.14020834863185883, 0.0802011489868164, 0.046848174184560776 ], [ 0.10126226395368576, 0.08568456023931503, 0.011780144646763802 ], [ 0.04818765074014664, 0.014645327813923359, -0.02486075460910797 ] ], [ [ 0.8083459138870239, 0.8701810836791992, 0.4228171706199646 ], [ 0.8594750761985779, 0.9532227516174316, 0.436838299036026 ], [ 0.45302775502204895, 0.4736107885837555, 0.20611847937107086 ] ] ], [ [ [ 0.058796171098947525, 0.08335628360509872, -0.0711517184972763 ], [ 0.0573323555290699, 0.21111132204532623, -0.03765500336885452 ], [ -0.033855997025966644, -0.09221343696117401, -0.05786622688174248 ] ], [ [ 0.6328456401824951, 0.8780357241630554, 0.32857784628868103 ], [ 0.7684171795845032, 1.0956979990005493, 0.4332287013530731 ], [ 0.42248716950416565, 0.45601850748062134, 0.17461632192134857 ] ], [ [ -0.7750219106674194, -0.8816672563552856, -0.32532528042793274 ], [ -0.9552940130233765, -1.00095534324646, -0.3210221827030182 ], [ -0.4230894446372986, -0.41579219698905945, -0.1073332279920578 ] ] ], [ [ [ -0.46355435252189636, -0.2625730037689209, 0.03156619891524315 ], [ -0.05102786049246788, -0.146687850356102, 0.07387083768844604 ], [ 0.20603887736797333, 0.0518198199570179, 0.004210569895803928 ] ], [ [ -0.625877857208252, -0.7585473656654358, -0.07889989018440247 ], [ -0.337210088968277, -0.6023625135421753, 0.019004467874765396 ], [ 0.20417436957359314, 0.01892290636897087, 0.1541927605867386 ] ], [ [ -0.2109488993883133, -0.1699456125497818, 0.09547708183526993 ], [ 0.014258615672588348, -0.0939343050122261, 0.15772822499275208 ], [ 0.2345331460237503, 0.03321376070380211, 0.04866067320108414 ] ] ], [ [ [ -0.08066356182098389, 0.06699484586715698, -0.05335163325071335 ], [ 0.137163445353508, 0.07116510719060898, -0.06270071119070053 ], [ -0.06939078122377396, -0.04035351425409317, -0.028031544759869576 ] ], [ [ -0.06085171177983284, 0.07180242985486984, -0.07156968861818314 ], [ 0.05532224103808403, 0.05118855461478233, -0.03401333466172218 ], [ -0.03794176131486893, -0.05365779623389244, 0.0020153732039034367 ] ], [ [ 0.5407347679138184, 0.5712735652923584, 0.126934215426445 ], [ 0.7878959774971008, 0.7312890291213989, 0.16087837517261505 ], [ 0.13191452622413635, 0.12024389207363129, -0.02063019573688507 ] ] ], [ [ [ 0.1828603297472, 0.3707127571105957, 0.09805241227149963 ], [ 0.19478116929531097, 0.481236070394516, 0.062155742198228836 ], [ 0.0438610278069973, 0.042197905480861664, 0.033447228372097015 ] ], [ [ -0.7129276990890503, -1.2270729541778564, -0.32946935296058655 ], [ -0.9773596525192261, -1.42634916305542, -0.285745769739151 ], [ -0.17941023409366608, -0.2296290248632431, -0.16497860848903656 ] ], [ [ 0.7074711918830872, 1.0014268159866333, 0.21635089814662933 ], [ 0.7776404023170471, 1.0897234678268433, 0.24309903383255005 ], [ 0.15569670498371124, 0.17921270430088043, 0.05190783739089966 ] ] ], [ [ [ 0.6448550820350647, 0.7757762670516968, 0.265536904335022 ], [ 0.855246365070343, 1.1213781833648682, 0.3977590799331665 ], [ 0.42601892352104187, 0.4510995149612427, 0.13173000514507294 ] ], [ [ -0.7698758840560913, -0.9989760518074036, -0.36500924825668335 ], [ -0.9064593315124512, -1.2775191068649292, -0.3780517578125 ], [ -0.32685261964797974, -0.3724842369556427, -0.142035111784935 ] ], [ [ 0.10142450779676437, 0.049159713089466095, 0.062290340662002563 ], [ -0.06228981167078018, -0.08224049210548401, -0.010450776666402817 ], [ 0.042704418301582336, 0.020593175664544106, 0.06695406883955002 ] ] ], [ [ [ -0.2333219051361084, -0.05861935392022133, 0.017658160999417305 ], [ 0.2726060152053833, 0.002925069769844413, -0.012226058170199394 ], [ 0.08304442465305328, -0.04444706439971924, -0.012449493631720543 ] ], [ [ -2.6253952980041504, -0.07974566519260406, -0.07114512473344803 ], [ 2.6212098598480225, 0.03813294321298599, 0.08351016044616699 ], [ 0.09143855422735214, -0.03474245220422745, -0.0011877852957695723 ] ], [ [ -0.12611427903175354, -0.0277447160333395, 0.0174252912402153 ], [ 0.13798558712005615, 0.02108697034418583, -0.025645820423960686 ], [ 0.06430352479219437, -0.03507714346051216, 0.0019475150620564818 ] ] ], [ [ [ -0.09095511585474014, -0.21872882544994354, -0.06964129209518433 ], [ -0.1523389220237732, -0.4586048424243927, -0.1487767994403839 ], [ -0.03353796526789665, -0.06752968579530716, -0.06365496665239334 ] ], [ [ 0.49509748816490173, 0.9233545660972595, 0.21358178555965424 ], [ 0.7725897431373596, 1.5698148012161255, 0.38938605785369873 ], [ 0.1232072189450264, 0.24568314850330353, -0.020314032211899757 ] ], [ [ -0.4995216727256775, -0.7280101776123047, -0.1727304756641388 ], [ -0.6577193140983582, -1.1066330671310425, -0.2805374562740326 ], [ -0.1264062374830246, -0.2190459668636322, 0.055434923619031906 ] ] ], [ [ [ -0.04300277680158615, -0.0655662938952446, -0.029654234647750854 ], [ -0.06257188320159912, -1.0924845933914185, -0.060749709606170654 ], [ -0.06638696044683456, -0.11511973291635513, -0.02709292061626911 ] ], [ [ 0.007220647297799587, -0.016386747360229492, 0.0067110443487763405 ], [ 0.02901639975607395, -0.7643899917602539, -0.05630763620138168 ], [ -0.013474308885633945, -0.10103007405996323, 0.010158869437873363 ] ], [ [ 0.03740652650594711, -0.0008063737186603248, 0.036541346460580826 ], [ -0.02309894561767578, -0.3653051555156708, -0.0035656869877129793 ], [ 0.02682569809257984, -0.02334381826221943, 0.032951291650533676 ] ] ], [ [ [ -0.6283855438232422, -1.0346978902816772, -0.4618341624736786 ], [ -0.990109384059906, -1.3493326902389526, -0.5677844882011414 ], [ -0.440218061208725, -0.5083809494972229, -0.22494329512119293 ] ], [ [ 0.5865362882614136, 0.9011220932006836, 0.4359624683856964 ], [ 0.8290916681289673, 1.1895203590393066, 0.5677972435951233 ], [ 0.44348475337028503, 0.4978010356426239, 0.2861088812351227 ] ], [ [ 0.22518539428710938, 0.24377207458019257, 0.18839772045612335 ], [ 0.23825781047344208, 0.24001827836036682, 0.10789249837398529 ], [ 0.12714509665966034, 0.12732839584350586, 0.04583052918314934 ] ] ], [ [ [ 0.14288341999053955, 0.1661665439605713, 0.017716268077492714 ], [ 0.21468505263328552, 0.4363313615322113, 0.15240894258022308 ], [ 0.02158236876130104, 0.1273614466190338, -0.07062479853630066 ] ], [ [ 0.22774554789066315, 0.12395881116390228, 0.1334252804517746 ], [ 0.2157345414161682, 0.07453242689371109, 0.15068846940994263 ], [ 0.10964358597993851, 0.17018309235572815, 0.1537860631942749 ] ], [ [ -0.31842097640037537, -0.48083704710006714, -0.07941428571939468 ], [ -0.49105679988861084, -0.7597746849060059, -0.35145196318626404 ], [ -0.19168363511562347, -0.2859923243522644, -0.10497064143419266 ] ] ], [ [ [ -0.23247992992401123, -0.16225722432136536, -0.0362224318087101 ], [ -0.24673308432102203, -0.17584271728992462, -0.007466368842869997 ], [ -0.04854286462068558, -0.11079102754592896, 0.0030275911558419466 ] ], [ [ -0.664275050163269, -0.41647112369537354, -0.11373399198055267 ], [ -0.7338221669197083, -0.480450838804245, 0.020727945491671562 ], [ -0.18460418283939362, -0.11569029837846756, 0.022433390840888023 ] ], [ [ -0.09195003658533096, -0.09647324681282043, -0.04394334927201271 ], [ -0.0414799265563488, -0.08566100895404816, 0.010918430052697659 ], [ -0.04652433097362518, -0.039986759424209595, 0.02866440825164318 ] ] ], [ [ [ -0.08907525986433029, -0.5057278871536255, -0.08514060825109482 ], [ 0.007414326071739197, 0.5591298937797546, 0.06905894726514816 ], [ -0.025935346260666847, 0.027297768741846085, -0.025238165631890297 ] ], [ [ 0.20276440680027008, -1.336762547492981, -0.12318651378154755 ], [ -0.1847866177558899, 1.3915544748306274, 0.09633810818195343 ], [ -0.015326730906963348, -0.10923270881175995, -0.07494840025901794 ] ], [ [ -0.06998037546873093, -0.2446768879890442, -0.08565784990787506 ], [ -0.029606150463223457, 0.32551082968711853, 0.06208110973238945 ], [ -0.032088473439216614, 0.017849473282694817, -0.02085847593843937 ] ] ], [ [ [ 0.09651488810777664, 0.23205925524234772, -0.08183037489652634 ], [ -0.029768701642751694, 0.14294277131557465, -0.05942956730723381 ], [ -0.0415957011282444, 0.112301766872406, 0.12354004383087158 ] ], [ [ 0.5116007924079895, 0.8406188488006592, 0.019703593105077744 ], [ 0.08330079168081284, 0.6935448050498962, -0.007170010358095169 ], [ 0.060804642736911774, -0.03487224876880646, 0.07630860805511475 ] ], [ [ 0.1472375988960266, 0.10639087855815887, -0.014882327057421207 ], [ 0.09490340203046799, 0.17497366666793823, -0.07323182374238968 ], [ -0.028872594237327576, 0.07445451617240906, 0.11657644063234329 ] ] ], [ [ [ -0.7229280471801758, -1.2723137140274048, -0.2044282853603363 ], [ 0.3907982110977173, 0.36343222856521606, 0.08707531541585922 ], [ 0.4489656388759613, 0.8129950761795044, 0.17313021421432495 ] ], [ [ -1.8849279880523682, -2.66172194480896, -0.8309814929962158 ], [ 0.359889954328537, 0.611809253692627, 0.1255803108215332 ], [ 1.5214461088180542, 2.21639084815979, 0.6208829879760742 ] ], [ [ -0.24490311741828918, -0.3867005705833435, -0.12782146036624908 ], [ 0.15008841454982758, 0.1402387022972107, 0.0680820569396019 ], [ 0.18753619492053986, 0.1987757831811905, 0.10601679980754852 ] ] ], [ [ [ 0.12476927787065506, 0.09450821578502655, -0.0021741928067058325 ], [ 0.23859353363513947, 0.4203175902366638, 0.05808408558368683 ], [ 0.027211686596274376, -0.058450207114219666, -0.11174662411212921 ] ], [ [ 0.5247494578361511, 0.43590083718299866, 0.007708608638495207 ], [ 0.48485320806503296, 0.7540436387062073, 0.13374803960323334 ], [ -0.0437060222029686, -0.13196393847465515, -0.1502121388912201 ] ], [ [ 0.06071457639336586, 0.02188105881214142, 0.02326376922428608 ], [ 0.1268029510974884, 0.12796497344970703, 0.021176433190703392 ], [ -0.011505565606057644, -0.044241104274988174, -0.07049383223056793 ] ] ], [ [ [ 0.046929191797971725, 0.005567304790019989, 0.11009315401315689 ], [ 1.1094356775283813, 0.06104682385921478, 0.001989752519875765 ], [ 0.13448961079120636, -0.029986880719661713, 0.06342284381389618 ] ], [ [ -0.02683747559785843, 0.07784740626811981, -0.0380820594727993 ], [ -0.05361492559313774, -0.009224426001310349, -0.025265684351325035 ], [ -0.04171985760331154, 0.016504906117916107, -0.010941273532807827 ] ], [ [ -0.012030511163175106, -0.026031576097011566, -0.01272142119705677 ], [ -0.05199361592531204, -0.003316761925816536, 0.0119745759293437 ], [ -0.04280630871653557, 0.017966631799936295, -0.025569753721356392 ] ] ], [ [ [ 0.05673861503601074, -0.3442171812057495, 0.29814037680625916 ], [ 0.07890255004167557, -0.3074108362197876, 0.2072567343711853 ], [ 0.03108009323477745, -0.06109263002872467, 0.007638668175786734 ] ], [ [ -0.05523533746600151, -2.350207805633545, 2.323551654815674 ], [ -0.04087957367300987, -2.1407501697540283, 2.204890251159668 ], [ 0.04314588010311127, -0.03548639640212059, -0.01842305436730385 ] ], [ [ 0.020133597776293755, -0.18985797464847565, 0.14316506683826447 ], [ 0.05472446233034134, -0.17922285199165344, 0.11863597482442856 ], [ 0.02496812306344509, -0.054119326174259186, 0.015111591666936874 ] ] ], [ [ [ -0.36826759576797485, 0.05346129834651947, -0.013111489824950695 ], [ 0.24502187967300415, 0.10179971158504486, 0.018803991377353668 ], [ 0.052893638610839844, 0.05131584778428078, -0.03583928197622299 ] ], [ [ -0.2062772661447525, 0.2702265977859497, 0.041720759123563766 ], [ -0.10007712244987488, 0.2137908637523651, 0.04379701986908913 ], [ 0.022239530459046364, 0.08246369659900665, -0.04936861991882324 ] ], [ [ -0.25231802463531494, 0.011950662359595299, -0.009359505027532578 ], [ 0.31619271636009216, 0.031017551198601723, -0.05029178410768509 ], [ 0.06575247645378113, -0.036072805523872375, -0.1268322914838791 ] ] ], [ [ [ -0.18031398952007294, -0.3078305423259735, -0.016197869554162025 ], [ -0.1326381117105484, -0.314590722322464, 0.04581848531961441 ], [ 0.001434437115676701, -0.023663945496082306, 0.029290681704878807 ] ], [ [ -0.4272288978099823, -0.7436265349388123, -0.047925107181072235 ], [ -0.26583242416381836, -0.7907370924949646, 0.04620717465877533 ], [ 0.004439732991158962, -0.0781710296869278, 0.1213066577911377 ] ], [ [ -0.10199767351150513, -0.14107772707939148, 0.011785715818405151 ], [ -0.0456390306353569, -0.13385623693466187, 0.017242588102817535 ], [ 0.002430260181427002, -0.026987910270690918, 0.04776272177696228 ] ] ], [ [ [ -0.013949154876172543, 0.03946760296821594, -0.01679888367652893 ], [ 0.03412811830639839, 0.7154437899589539, 0.04671855643391609 ], [ -0.015230119228363037, 0.04908209666609764, -0.023905234411358833 ] ], [ [ -0.0017834395403042436, 0.05842486023902893, 0.029108626767992973 ], [ -0.00008539984264643863, 0.24045227468013763, 0.008705812506377697 ], [ 0.01104763988405466, 0.08698895573616028, 0.0353543646633625 ] ], [ [ 0.014890934340655804, 0.0023694531992077827, -0.016901109367609024 ], [ 0.011412251740694046, 0.36613452434539795, -0.015485808253288269 ], [ -0.040096547454595566, -0.06679188460111618, -0.025204453617334366 ] ] ], [ [ [ 0.21904590725898743, 0.568728506565094, 0.07251344621181488 ], [ -0.09646545350551605, -0.6750847697257996, -0.09228291362524033 ], [ 0.05512077733874321, -0.09400464594364166, 0.008183701895177364 ] ], [ [ 0.13221536576747894, 1.3762810230255127, 0.03968409076333046 ], [ -0.09568770974874496, -1.4706544876098633, -0.08553091436624527 ], [ 0.02074281871318817, -0.043814875185489655, 0.044483982026576996 ] ], [ [ 0.14102880656719208, 0.3832210600376129, 0.04342801868915558 ], [ -0.05153796821832657, -0.37412646412849426, -0.0797242671251297 ], [ 0.03593098372220993, -0.09172195196151733, 0.028739754110574722 ] ] ], [ [ [ -0.04175552725791931, 0.03395365551114082, 0.006025323644280434 ], [ -0.017435219138860703, 0.15353664755821228, 0.03521445766091347 ], [ -0.02121744304895401, 0.03729405254125595, -0.013773733749985695 ] ], [ [ 0.008787543512880802, 0.0701339989900589, 0.027702312916517258 ], [ 0.1079016700387001, 2.0528509616851807, 0.03867277130484581 ], [ 0.031473398208618164, 0.05735735967755318, 0.016289005056023598 ] ], [ [ -0.03804020211100578, 0.03467007726430893, -0.024132180958986282 ], [ 0.0001444255030946806, 0.32950520515441895, 0.017702477052807808 ], [ -0.03270495682954788, 0.017887180671095848, -0.03365739807486534 ] ] ], [ [ [ -0.03998103737831116, 0.06323116272687912, 0.009569073095917702 ], [ -0.4411902129650116, -0.38171955943107605, -0.03586957976222038 ], [ 0.47440284490585327, 0.27364158630371094, 0.006098921410739422 ] ], [ [ -0.037365417927503586, -0.048531826585531235, 0.0796576589345932 ], [ -2.3132498264312744, -2.153818368911743, -0.06571041792631149 ], [ 2.241406202316284, 2.201124906539917, 0.03273104876279831 ] ], [ [ -0.0036466571036726236, 0.022369517013430595, 0.02207104302942753 ], [ -0.15473444759845734, -0.19669309258460999, -0.04969623684883118 ], [ 0.2113747000694275, 0.10266326367855072, -0.01015976071357727 ] ] ], [ [ [ -0.36172571778297424, -0.03300758823752403, -0.1474684327840805 ], [ -0.062240809202194214, 0.5624579191207886, 0.07320359349250793 ], [ -0.22507846355438232, 0.14275968074798584, 0.056338150054216385 ] ], [ [ -2.554978132247925, -0.20979070663452148, -0.09245805442333221 ], [ 0.08752262592315674, 2.512366533279419, 0.11505569517612457 ], [ -0.11767906695604324, 0.22719387710094452, 0.03759007900953293 ] ], [ [ -0.18318217992782593, 0.061619628220796585, -0.07983869314193726 ], [ -0.0391593836247921, 0.23017121851444244, 0.0532136894762516 ], [ -0.14887578785419464, 0.0837387889623642, 0.01188741810619831 ] ] ], [ [ [ -0.3290765583515167, -0.1621301919221878, 0.016724800691008568 ], [ -0.2662808895111084, -0.11034732311964035, 0.01069191936403513 ], [ 0.04809742420911789, 0.06221426650881767, 0.015402339398860931 ] ], [ [ -0.6337690949440002, -0.4286348223686218, 0.07476869970560074 ], [ -0.7615185976028442, -0.5187259316444397, 0.07840161770582199 ], [ -0.03377047926187515, 0.16467365622520447, 0.10605861246585846 ] ], [ [ -0.16727682948112488, -0.06927219033241272, 0.006499067414551973 ], [ -0.1113610491156578, -0.03952357545495033, -0.0021834103390574455 ], [ 0.04850565269589424, 0.04377703741192818, 0.04426499456167221 ] ] ] ]

[lzrvch]

The ONNX model you exported would contain weights in FP32 precision but also additional Quantizer nodes in the model graph - FakeQuantize nodes (or QuantizeLinear/DequantizeLinear node pairs, if you enabled that option). Weights in the original precision would be passed through those quantization nodes first and you would get the discretized weight tensor as a result. You should also have such quantizer nodes for the activation tensors. You can check the presence of these nodes with a model graph visualization tool like Netron.

OpenVINO runtime can interpret a model graph with these quantizer nodes and execute the actual convolution operations in INT8 precision efficiently.

[YoungSharp]

After using Netron to visualize my onnx network I can't find any FakeQuantize nodes.
Can you tell me how to enable those option to get FakeQuantize nodes?
input_size = 448 ENABLE_COMPRESSION = True nncf_config = dict(compression=[dict(algorithm='quantization', initializer=dict(range=dict(num_init_steps=10)))], log_dir=work_dir)
is current used config.

[RanyaJumah]

Hi!
I also got a bit confused about the saved compressed model since I used different techniques, and for example for the filter pruning one the model size still the same after the fine-tuning process and the inference time it incresed a bit (~2 ms) but when I checked the wieght values it changed, do you have any idea why this happend and just to clarify what I did ( I load the pre-trained model {Encoder-Decoder) and I compress the encoder part and use the wrapped model for fine-tuning the whole model) ?

[lzrvch]

@YoungSharp FakeQuantize nodes should be inserted in the model graph automatically upon the execution of the wrap_nncf_model function in the mmdetection patch we provided, since you specified the quantization algo in the config. You can notice the print(*get_all_modules(model).keys(), sep="\n") statement right after wrap_nncf_model is executed. You should see the names of all the modules of the model that are wrapped by nncf in your output. Also you should be able to see that info in the logs written to the work_dir folder.
Did you modify the mmdetection patch from third_party_integration/mmdetection beyond the config specification? Could you please provide, if possible, the patch you used for mmdetection, in case you modified it, and the config used so that we could reproduce your issue?

[lzrvch]

Hi @RanyaJumah!

Regarding your question on filter pruning - the .pth checkpoint file of the pruned model that you get after fine-tuning contains additional binary masks that are used to determine which filters to zero out, along with the model weights of their original shape. Hence the .pth file is similar in size to the original one. Now when you infer this model, the masks are being applied, but you still have these zero filters that are used in the convolution operations.

In order to get an actually compressed model with fewer FLOPs, we provide capabilities for ONNX export via the export_model method of the compression controller (see #29 for details). The zero filters are discarded there upon the model export, so that the weight and activation tensors actually change their shapes.

If you wish to infer your compressed model in PyTorch directly, you can simply modify the export_model method of the FilterPruningController, so that the redundant filters and masks are discarded, but the PyTorch model is returned. If you will use this PyTorch export, we welcome you to open a PR and contribute this capability to the main codebase.