Mandatory Systeminfo 5.56 Update and new benchmark rules for RTX40 series
New Systeminfo Update for UL Benchmarks and ECC memory state for RTX 40XX user requirement
Due to the inflated and bugged scores we have seen lately pop up at the UL Hall of Fame and at HWBOT it would be recommended to enable the ECC Memory setting in the nVIDIA Control Panel. This is a request for testing to see if this is a solid fix by UL.
However just enforing this in a screenshot would not be sufficient. Therefore UL Benchmarks has implementend ECC detection in their latest Systeminfo release 5.56
- Therefore only use systeminfo 5.56 or newer to be used from the 23 November 2022. Note that HW monitoring must be enabled
Existing scores will be moderated accordingly by the HWBOT moderation Team as the one from UL benchmarks.
- ECC needs to be enabled in the nVIDIA control panel (Testing purposes)
All the related UL 3D Benchmark Rules will be updated accordingly
RTX 4090 ECC enable 3DMARK - SPEED WAY - 9261 https://www.3dmark.com/sw/107645 RTX 4090 ECC disable 3DMARK - SPEED WAY - 9909 https://www.3dmark.com/sw/107672
Hm that is actually a pretty interesting thing. How is this happening only with 4090 but not with previous generations?
says:
Can of worms on it's way .
Provide an alternative, don't rely on benchers honesty, we all know it won't work...
My 2 cents: We need proper build in monitoring tools from UL and have them automatically invalidate scores if the FPS jump out of bounce.
Nailed it
Is this why you invalidated a bunch of my valid ran benchmarks?
says:
If that is the case , should be lot more on the horizon with invalid scores .
Glad I got rid of the 4090 and I am only benching "legacy" type cards .
Maybe all 4090 GPU submissions should be invalidated until all of the details are worked out and then everyone benching a 4090 can start off with a clean slate and level playing field rather than arbitrary selection of scores based on an individual's impression of the result.
says:
What happens to people who bought a very expensive card before 23 November 2022 and posted scores ?
Mr.Fox idea not going to make a lot of people happy but if that is what you decide , a clean sweep , I'll go for it.
Can of worms though.
Then nobody can say they're being treated differently, scrutinized through biased eyes or having their scores treated unfairly. I mean, if it is really that important to make a new rule that applies to a new niche product, it would probably add value to have only clean data as well as not generating additional angst and controversy. If the people with high scores that get wiped out were submitting legit benchmark runs with reliable hardware they can just do it again and resubmit. Not the end of the world, and certainly removes any questions or doubt about fairness and rules applying equitably to all. Until UL fixes 3DMark and updates SystemInfo to determine when ECC is enabled/disabled, nobody gets to submit 3DMark benches. Uncomplicated and fair, and no basis for controversy or an appearance of loose and subjective rule enforcement that varies by user name.
I guess 4090 owners are not the only early adoption casualty. Team Red has their own drama going on now, too.
https://github.com/Mesa3D/mesa/blob/2ed9eb1b633f214ff8900ab3be9e639f87cebaef/src/gallium/drivers/radeonsi/si_state_shaders.cpp#L671
Then delete all 4090 UL based benchmarks and start over with new rules. Let UL handle their rankings in their own way. For those that borked their cards in the process of these inflated over the top scores will just need to buy another card and try again. LOL
says:
Yea , we all got thousands to spend on another card ?
Last thing we want to do is put people off benching .
Have to come up with something else rather than just deleting ALL 4090 scores .
I am just a soldier , it is up to the generals ( Alby etc.)
What are you talking about? This is like the 4th time they got me with hardware I no longer own. Didn't see anyone waving the white flag when I got hit laptop benching. And laptop benching is like 5 to 10 times more expensive! So pony up or except the loses like I and others already have had to do.
With my little experience of these 4090, only 3 benchmarks can be strongly improved by unstable memory : PortRoyal, Superposition 1080p and Superposition 4K the others not Perhaps only these 3 benchmarks would merit a requirement to enable ECC and thus remove all current scores and thus allow ECC to be disabled for all other benchmarks. This would make everyone happy I think and would be quite fair for all
says:
If we are going to "re-do old scores" I vote that we take the time and start invalidating scores that don't even pass 3dmark muster like LOD/Tess scamming Firestrike scores. If I am really dreaming, I vote we stop allowing old and archaic benchmarks being run on current gen systems period. My thoughts are it should be a hardware OC site not a software OC site where because people who know old ass benchmark nuances from Windows XP era can rack up points on current generation hardware not due to actual hardware being different but simply because they know how to maximize settings to get good scores on old ass benchmarks. Those old benchmarks should still provide points for old hardware but it doesn't seem the same running 4090 cards on 3DMark 2001 SE just because people know how to game the software more than the hardware. Just my two cents and Im sure these comments will rouse some feelings.
HWbot doesn't have 4K....
Side note:
More wasted money! 3090 Nvlink SLI
says:
Lets have a formula 1 race where we all use the same cars ?
What is the point ?
BTW Legacy is not going anywhere except here , forever ?
says:
The formula 1 example you used couldnt be worse, in Formula 1 all cars must adhere to specific rules. Its not like in Formula 1 Lewis Hamilton is using an F1 car and Verstappen is using a Subaru Wagon.
says:
You are saying lets get rid of the fastest cars because I don't understand them and how to make them work ?
Point made I think , have fun and out of here .
says:
Also in a Formula 1 race we all use similarly regulated cars and we are all on the same track. Just because someone knows old software shortcuts that improve hardware scores from 20 years ago shouldn't provide bonus points on current generation hardware that would literally never be used on those old OSes. Who is running Windows XP for anything other than benchmarks in 2022? Why are we testing 2022 hardware and giving points on that hardware for software that won't even utilize it. Having to disable features of your card like Ray tracing to game the benchmarking is just that, software tweaking. Thats not hardware overclocking, thats software maximizing. All these shenanigans do is confuse anyone who wants to get into overclocking in 2022 and beyond. I am of the opinion that we shouldn't have to tweak a 20 year old OS and run a 20 year old benchmark just to rack up a few HWbot points in 2022 and I will continue to lobby for current tech to use current, ungamed tests.
says:
I never said "lets get rid of the fastest cars because I don't understand them and how to make them work" and I'd appreciate it if you either quote me or stop making up stuff. I see enough of that elsewhere, and I'd appreciate not having to do deal with it on a site where we literally have to prove what we are submitting. Thanks and god bless.
No wonder this sport is dying. Going Stasi-psycho-chihuahua because of a few random cheaters because it threatens a small group's ability to claim dominion is enough to make a lot of people (the peasants) say to hell with all of it. If this is the new approach I think all of the submissions from the least to the greatest should be wiped out completely rather than taking guesses at who might be cheating and who isn't and arbitrarily invalidating the scores of someone that "must be a cheater" because they are not a known member of the HWBOT Royalty.
The last 20 years. Either oc is about the process of dying or it is zombie already:)
says:
It's not that bad .
Just because someone complains about Legacy .
As far as the 4090 goes , if it will sort things , I would rather see every score wiped because it seems to be affecting every other bench.
I have a lot to lose but I would rather see fair benching carry on and lose my points.
Maybe best to scrub 4090 full stop until new rules which are MANDATORY are in place .
By MANDATORY I mean , follow the rules or get a warning then a vacation .
We are still waiiting for the 4080 not able to enable ECC issue to take a final decision/ However in my eyes and stated already several times bnefore, it is up to the benchmark developers that need to take action and rectify the flaws in their system monitoring and Validation check.
Hopefully the new GPUZ version will show ECC enabled or not... that would already be something in the right direction for older benchmarks
says:
Is ECC only applicable for 4090 and later models Alby ?
Are you guys going to finish removing invalid 4090 benchmarks or not? I am starting the process of redoing them and trying to get the ones of mine in question deleted, but if that is not the case then I do not want to potentially have good scores deleted.
We will not remove any benchmarks from HWBOT for the RTX 40 series... submission wise we are still hoping for another UL fix and the GPUZ implementation....
This actually doesn't make any sense. So you are keeping the BS runs while new submitted runs are under scrutiny of the new ruling? Which makes them lower and slower runs unable to beat the bugged runs. That sounds very productive.
Once more we will not remove benchmarks for RTX 4090, we will remove/re-enable submissions only after we get a clear view of what we can do to aid monitoring/moderation.
Not sure what doesn't compute here...
Hi bro. I think the point John was making is someone actually did to him what you keep saying will not be done. I think it happened to others as well. So, you are saying you're going to stop doing that now because doing so was a mistake, or you were not aware that had already taken place?
Have a Merry Christmas and Happy New Year.
The scores John got removed where 3090 SLI, is 4090 scores are all there even is port royal.
Talking about before we even got to that point.
This is the answer. Yes atm both ECC on/off score are on hwbot, from before and after the change of rules. We are just waiting for a fix to take a final decision and yes everyone will have to follow the same rule, no mix of both.
We will slowly start removing the newer RTX 4090 scores with ECC disabled, this to ensure user with new subs get a more clearer signal. We have observed some shenanigans again by users bending the rules for their own benefit...
Our advice is to not bench these cards on LN2 or other extreme "expensive" cooling
UL is still on it and monitoring...
Hi. I saw one of you have blocked my 4090 Fire Strike Extreme subs. Reason. ECC disabled run. But I'm quite sure I had it Enabled. I don't even think older Fire strike benchmark offer driver status as ECC disabled/Enabled in their results summary. Or maybe LOD tweaks is one of the culprits (allowed for hwbot) for not show it up. I can't confirm this right now as I have returned the card. Maybe others can test out this with 4090. I mean every single Fire Strike Extreme benches posted on hwbot don't have the results summary with ECC disabled or Enabled in their results. Either it will be showed as Driver version approved or the driver version. Nothing about ECC disabled or Enabled. What to do with results that is valid but you can't show it due Futuremark system info can't show it correct? Will all 3DM Suite Fire Strike (Extreme) results be removed/blocked now even if they are done correct? Thanks
In short... Newest Futuremark System info 5.55 or newer doesn't matter. Offer nothing about ECC disabled or Enabled info for some benchmarks as etc 3DM Suite Fires strike Extreme or Fire Strike. You either get approved or the driver version instead in the results summary. Nothing more. Correct me if I'm wrong. It will be wrong blocking posted results if this is correct and you haven't done something or anything wrong. Examples below.
Edit. Same here from today.... https://www.3dmark.com/3dm/87613114 ------ Newest Futuremark System Info, Newest drivers, software and nothing about ECC info in the results summary https://hwbot.org/submission/5178098_andressergio_3dmark___fire_strike_extreme_geforce_rtx_4090_43526_marks
I have even seen results from 3Dmark TIME SPY flagged as ECC disabled even if the settings is enabled in NCP. What a fun time we live in.
Edit. And what with 4080 results? Not sure ECC enabled/disabled will show up at all for this 4000 series cards.
That really stinks. If they are going to remove 4090 submissions, then they should just remove all of them across the board and make everyone start over rather than making arbitrary assumptions about which ones are legit and which ones are not.
Example: https://hwbot.org/submission/5130021_splave_3dmark___time_spy_extreme_geforce_rtx_4090_21023_marks
https://www.3dmark.com/3dm/83861241
Why is this still on the leaderboard and receiving points? But others are removed?
Remarks :
- when did we impose the ruling? We are blocking the ones atm that don't adhere to this rule set. Anyway did you think you could get away with a small
increase in freq that would boost your score so high that you even beat LN2 or chilled setups... now some opt that the best defence is to attack...
- What is the safest score to submit, the ECC or the non ECC one ? We see user sub ECC enabled ones for the non global benchmarks and trying
to stay under the radar and subbing ECC disabled ones for the benchmarks with global points.
These are the ones why this is done now ! Thank them not me!
- Is it THE requirement that ECC needs to be visible in the UL verification link to determine it was enabled or not? Maybe we just need a safety net
- UL is going to rework ECC detection for older benchmarks
- If UL decides to go with ECC enabled only is the way to go, the older scores will be removed and will need to be rebenched...
- if you have a more viable solution for this mess, plz share it...
- If you have a score that you think was legit, link us, maybe we messed up. Moderators are only humans too.
End of drama
Been writing support tickets for months now begging and asking for help to fix my forum account, apparently password restoration function is not working and nothing is being delivered to gmail associated with hwbot account. Can not even communicate on forum and ask questions there. Can not see submissions being removed, or the reason for that, unless manually going throughout my submissions and noticing them just disappear. Just like now went to check up my points and noticed them drop, and sure enough them some RTX 4090 scores got removed again ! Leeghoofd i am asking can you or any admins/mods please help with fixing the forum password and look up why i am not receiving notifications to my gmail for submissions being removed ? And what was wrong with these submissions and the reason for them being removed ? Link bellow, thank you :) : https://hwbot.org/submission/5174960_ https://hwbot.org/submission/5174958_ https://hwbot.org/submission/5174956_
why not just drop a PM on my wall or come on Discord... I'm confused as you are on the forum now. Plz mail me your credential to albrecht@hwbot.org
Wow so all 4090 benches will be deleted? What about the last ditch effort that EVGA did? Just poof gone? RIP.
No eVGA scores to my knowing upped to HWBOT from them limited edition cards
A suggestion... if ECC enablement is required, perhaps also require NVIDIA Control Panel in the screenshot. I purchased a 4090 since this debacle started (received Friday) and all of my submissions have it enabled. I see many older submissions (before ECC checking was a thing) often with lower core and memory clocks than I am running with signficantly higher benchmark scores. If I disable ECC, I can match them, but don't see any point in uploading 3DMark runs only to have them deleted later. It is a shame that we have to gimp performance by having this enabled, but I suppose if everyone has to gimp their 4090 it will level the playing field. I have ECC disabled when I am not submitting scores for the leaderboard. I kind of wish the option was not even present in NVIDIA Control Panel.
Example of my suggestion:
Why does one continuously keep on repeating the same thing.
All older non ECC scores will be removed on the 15th of this month... So everything will be the same. I can add an ECC easily in the screenshot, no special Photoshop skills required... Doesn't work
If Nvidia removes ECC we have a big problem, than it's fully up to UL to impose proper bugged score detection...as users will keep on claiming their score is fully legit and will cry it is labeled Valid by UL.
Impossible to moderate with the current graph data we have...
What is being repeated? I don't understand. I merely offered a suggestion to provide an additional point of reference just as the CPU-Z and GPU-Z are required in screenshots for validation purposes. But, we still require them even though you can Photoshop them. I took your invitation to offer a suggestion seriously. Perhaps I should have just kept my idea to myself. I apologize if it was a point of frustration, but I appreciate you taking the time to respond.
If NVIDIA had never added ECC as an option then we would not be having this a topic for discussion. It impairs performance on GPUs just as it does system memory with ECC. So does G-SYNC, which I also hate and always disable. And, bugged 3DMark scores have been a reality since the very first benchmark was made. I suspect that many submissions are bugged and the people submitting them, and those displaced by those scores, have no knowledge that they are.
Cheaters are going to cheat. I can Photoshop almost anything if I want to go to enough effort. There is nothing to be gained by cheating, at least when it comes to submitting scores on the leaderboard. No money to be made or anything else of substance. Cheaters have to live with the knowledge they are actually losers and trashy human beings that don't deserve the air that they breathe. Unfortunately, those worthless bastards make us have to jump through hoops to foil their foolishness. Their nonsense creates division and controversy among people that should enjoy friendships built around the hobby they have in common.
Why the smart ass reply? So it's like that around here is it?
Without ECC we would have bugged outputs all over the place with these RTX cards, especially if the benchmark still VALIDATES on the verification page. There should be an autodetect system monitoring weird outputs that would invalidate the score if eg Scene 3 scores more than 30% faster than scene 7... sadly this is not the case (yet)
Now with Pasi from UL monitoring the subs with ECC there are almost no bugged outputs. With UL displaying the ECC enabled and correct memory size we have a double backup layer from the standard HWBOT screenshot requirements. Adding ECC tab is overkill, it has no real added value
One member was already creative, making a new screenshot with ECC enabled but with the non ECC save profile loaded in the benchmark. The verification link gave it all away.
Can we judge by the displayed frequencies, nope but again we see them in the verification link. Users can downclock or even upclock to make it look more realistic. It's that creativity that leads to so many rules and rule changes...
Just wait for the 15th, when we pull all non ECC subs, the dust will settle after a few days... hopefully everybody is satisfied with the new direction...
Fingers crossed UL wil pickup the feedback for future benchmark releases, the current verification design has flaws. We saw it already with 6900XT scores and now once more with RTX 4090. Sadly no site is perfect as way too many "tweaked" scores stay under the radar...
Now this was more along the lines of what I was expecting from you. Thanks for the info.
@Mr. Foxanother perfect example https://hwbot.org/submission/5200989... ECC on, ROFL
Are you guys slowly starting to understand now what we deal with each day...
They're wrapping themselves up, they don't even know what the hell to invent. how do you do it now? Many in the top rankings would see positions torn apart, a big problem arises here. HWBOT at historic lows. LOL ?
Yes, I understand what the problem is.
In fact, here is another twist. I just submitted a bugged run by mistake. NVIDIA Control Panel shows ECC is enabled, but GPU-Z shows a memory value that tells me it is NOT enabled. So, maybe a driver glitch. Please delete this. It should not have any points: https://hwbot.org/submission/5201294_
I rebooted and then ran another Firestrike right on its heels and this is a valid run. This one should stay. NVIDIA Control Panel AND CPU-Z both confirm ECC is enabled. This one should stay and get points.
https://hwbot.org/submission/5201316_ (but it shows no points because my bugged run is stealing them).
wel its simple if we spot 24GB it is out, bug or glitch or whatever you want to call it...
You have a better score: https://hwbot.org/submission/5201360
Yes, I ran that better one after posting. ?
I think what happened with the bugged submission is that a TDR kicked off the ECC enablement. Since this is all driver/software based and not baked into the vBIOS the chances still exist for it. Otherwise, I would not know how to explain it being enabled in Control Panel and not actually enabled. Weird stuff... lots of bugs and glitches with the new tech.
So since the other thread got locked.... When are you going to start removing scores from the ranks? Are you starting with global ranks first? And can we be alerted to which benchmarks are on the chopping block first. Thanks. Because the 15th has pretty much came and went on that side of the world, so just wondering.
It's already happening.
says:
Well, my bugged score in Speedway that I also flagged for deletion still blocks my valid ECC enabled run: link: https://hwbot.org/submission/5201824_
When will this be fixed?
Best regards!
says:
Thx for fixing! ?
One more left though, sorry for all the fuss!
https://hwbot.org/submission/5173687_udet1916_3dmark___time_spy_extreme_geforce_rtx_4090_18101_marks
I took a break from benching and did not recognize the change in rules regarding ECC - never had any visual artifacts and my scores were not near any LN2 or H2O stuff as far as I recognized, otherwise I would have reported immediately (just air cooling 4090 atm)
One question regarding the ecc rule, many benchmarks seem to not require ecc. 3d11 and vantage give globals but don't need ecc, how come? Also many older benchmarks without globals don't need ecc, like heaven, 3d03, 3d05 etc. Shouldn't all 3d benchmarks have the same rules, i would assume bugged runs can happen in those benchmarks also?
Only exception would be gpu-pi as it does not involve memory at all and it actually checks that calculations are correct.
Really good question there....
It this expected for all 3D graphics benchmarks or only 3DMark? I ask because I see lots of submissions for things like Catzilla with ECC disabled and the only way to compete with them is to disable ECC. Are those with ECC disabled at risk for moderator deletion like the 3DMark submissions?
We just imposed ECC in the 3DMark suite and the SuperPosition benchmarks (as they feature build in detection) older legacy benchmarks are not taken into account...
Thank you. I appreciate the clarification. Have a good week.
Please log in or register to comment.