[Edit:] I see the boinc messageboard echo in here works fine :)

Beg your pardon?

Double posts seem to happen a lot (to me anyway) [not this time]

Your resident moderator(s) will probbaly be pleased if you red-x them for hiding. That's tongue in cheek. For once it's not me getting those reports :D

Done!

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

OK, so it seems that my 2-hour dinner break was largely taken up by a discussion of initial <flops> for new hosts. That's fine: I think it's one of the most overdue refinements of CreditNew, so let's run with it for a while.

In the CPU case, do we agree that the Whetstone benchmark understates the scientific performance by some ratio R, where R is greater than one, and rises to somewhere around 3 or 4 in the AVX case?

And in the GPU case, do we agree that marketing 'peak FLOPS' overstates the scientific performance by some ratio r, where r is less than one, and may take a value of 0.1 or 0.01?

The question becomes, how do we choose values of R and r which are "good enough" to get a new cruncher started, without frightening him/her with infeasible runtime estimates, or risking his/her tasks being aborted for 'TIME_LIMIT_EXCEEDED'?

Jason has proposed a function MIN(SIMD_capability(CPU), SIMD_optimisation(app_ver)) for the CPU case. We need a curve for that...

Nobody seems to have studied the GPU case, but surely r is knowable? (A 'known unknown', on the Rumsfeld grid).

So, is r the same for AMD, NVIDIA and Intel? Is r the same for each major hardware version from each of the manufacturers? Is r the same for all project applications? Do we need to scale r by GPU utilization factor? We need a curve for that, too...

And once we have R and r "good enough" to get us started, it becomes a non-problem: provided they let us complete 11 tasks, APR takes over and seems adequate. Jason tells us that properly damped averages will take over from administrator responsibility for outlier management.

And now we need to talk about credit...

Yes, for initial scaling, new CPU host and/or app, we're talking a fairly simple matter of providing a coarse 'improvement'. The objectives for that as an initial test will be just to use existing information where available, such as Android's SIMD aware bench that was snuck in without the corresponding server side awareness. x86 (from FPU through AVX) will actually be slightly simpler since at that point the mechanism only needs relatively coarse numbers (explained later), and we have no SIMD aware bench to confuse the picture right now.

Picturing the Android case in the context of our recent SIMAP adventure makes things a bit simpler. For that Android case, a flag 'client_bench_is_simd_aware' would be set (by platform and cpu feature information readily available in the scheduler request. The missing part of that is a corresponding plan class spec setting defining the max supported simd vector length by the app. Correcting that gives the coarse (pre efficiency scaling) information necessary to put the raw pfc correction (think DCF) nack in the ballpark where averages can cope, the other key component being returned elapsed. Fine correction (for efficiency & utilisation) is for the averaging portion, and then adjusts the host_scale, then subsequent normalisation.

A take away from this point is that precise original project estimates are not required, any more than within some reasonable theoretically based 'minimal' number of operations estimate. If those were 'perfect', utilisation was 100% and users never used their hosts except for crunching, host_scale (think DCF again) would be 1 and not vary. That's obviously not a practical design goal from any standpoint. ... However these scales are registering as less than 1, which simply means the wrong numbers are being used ( and what's wrong is not , so far, the project supplied number of operations)

For x86 Since we (the scheduler) know the client version and platform, as well as the CPU features and application capabilities, again there's enough information to set flags draft a coarse correction. (until such time as a SIMD aware bench is crafted for x86, an easy server side switch to throw when appropriate)

That quite coarse correction is a major stabilising one simply by stopping claims of less than the minimum possible to do the tasks getting in. Those are the key claims that mess up several stages. Since such low claims do get in at the moment, and break the minimumal # of operations, fixing that in coarse fashion is the first step.

WHere SIMD bench is not available, *reasonable* SIMD correction to keep things in the right order of magnitude need only be as rough as existing Boinc Whetstone plus 50% per power of two of the vector length (x1.5^(logbase2(used_simd_vector_length). That gives coarse correction for a common example SSE-SSE4.2 being on average about 2.25x faster than a pure FPU implementation.


At this point the 'ideal' (not quite as implemented) flow of estimate, and where domain knowledge comes into play becomes the key. Localising Domain specific knowledge is one software engineering & control systems concept Boinc software has problems with throughout:

- Original task estimates: Theoretically derived, very coarse, never changes (unless the work does). has no knowledge of applications or hardware (and should never have). They only know about theoretical number of operations required to do some work (This bit *appears* fine at this stage, pending thorough checks for obvious omission of key computing steps
- Global scale (a bit less coarse, should change very slowly, on the order of 1000's of tasks): Only knows about a bunch of applications, and 'normalises' the performance scales (currently against the 'most efficient' app available for given tasks.
- Host Scale: (Think Server Side DCF) Has substantial application and host domain specific knowledge available that is currently misused. SHould be responsive to hardware, application and utilisation change without falling over.
(change on the order of 10's of reports
- Client side: Should be most responsive, is currently disabled &/or broken, probably on the misguided idea that this was upsetting the earlier steps.

What currently upsets the system, making stage 1 look worse than it is, is that steps 2 through 4 are seriously under-engineered and spaghettified. Control domains in stages 2 & 3 in particular overlap and interact, and process irrelevant noise that could be easily conditioned.

GPU has indeed been studied as well, but as we come from seti-land, step 2 cross application version scaling clouds the issue. The extent to which this cross-scaling is intertwined here is not yet clear, but GPU will require the reverse process to CPU correction.. stabilise first then coarse correct second. That's because the hardware is more diverse and ultimately (in the seti case at least) relies on the CPU end being stable.

Note that at no stage do any of these ammendments/refinements explicitly adjust the credit. They absorb design limitations inherent in the underdesigned - pverimplemented control system.

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

right, move some info over to th Wiki then...

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

right, move some info over to th Wiki then...

A lot of that's being documented right now, in an experimentation format. Copy/paste for the sakes of verifying/discussing might help. The general concepts there give way to pretty specific adjustments from here on.

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

right, move some info over to th Wiki then...

A lot of that's being documented right now, in an experimentation format. Copy/paste for the sakes of verifying/discussing might help. The general concepts there give way to pretty specific adjustments from here on.

If you write it there is a tendency for the faces of the general public to go blank, while their collective eyes glaze over.

You can check out mine, it's short.

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

right, move some info over to th Wiki then...

A lot of that's being documented right now, in an experimentation format. Copy/paste for the sakes of verifying/discussing might help. The general concepts there give way to pretty specific adjustments from here on.

If you write it there is a tendency for the faces of the general public to go blank, while their collective eyes glaze over.

You can check out mine, it's short.

I don't see it on the wiki yet ?

Mine's for the purpose of ensuring we don't all go off on Random directions just now. We established enough details to make pretty rapid progress, and stop and adjust at key points.

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

I don't see it on the wiki yet ?

Mine's for the purpose of ensuring we don't all go off on Random directions just now. We established enough details to make pretty rapid progress, and stop and adjust at key points.

You need to follow the 'Day zero observations' Link from the 'Observations from live run at Albert' section

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

I don't see it on the wiki yet ?

Mine's for the purpose of ensuring we don't all go off on Random directions just now. We established enough details to make pretty rapid progress, and stop and adjust at key points.

You need to follow the 'Day zero observations' Link from the 'Observations from live run at Albert' section

Thanks, perusing with a short break from my dictophone.

[Edit:] Yep should be able to tie objectives on the passes in nicely to that. Yeah I was aware of the logarithmic scales hiding a multitude of the sins, so it's great to see that awareness reflected in the non-log representation. Nose will be back to the grindstone a bit later.

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

[Edit:] Yep should be able to tie objectives on the passes in nicely to that. Yeah I was aware of the logarithmic scales hiding a multitude of the sins, so it's great to see that awareness reflected in the non-log representation. Nose will be back to the grindstone a bit later.

Did I mention faces going blank and eyes glazing over?

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

[Edit:] Yep should be able to tie objectives on the passes in nicely to that. Yeah I was aware of the logarithmic scales hiding a multitude of the sins, so it's great to see that awareness reflected in the non-log representation. Nose will be back to the grindstone a bit later.

Did I mention faces going blank and eyes glazing over?

At this stage the documentation is for our benefit. For many others the proof is in the pudding, and how we get there of little importance to them...

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

[Edit:] Yep should be able to tie objectives on the passes in nicely to that. Yeah I was aware of the logarithmic scales hiding a multitude of the sins, so it's great to see that awareness reflected in the non-log representation. Nose will be back to the grindstone a bit later.

Did I mention faces going blank and eyes glazing over?

At this stage the documentation is for our benefit. For many others the proof is in the pudding, and how we get there of little importance to them...

Yes, but "our benefit" includes the wider benefit of getting code changes accepted into the master BOINC codebase. Hand-waving demonstrations of "but look - it works" are simply descending to the level of the lowest common denominator. Recording and documenting the steps which led to the conclusions being implemented in code will help - even if only by giving us the warm fuzzies that come from occupying the moral high ground.

I've started documenting the wingmates who co-validate my 'high outlier' credit scores, but no pattern has emerged yet.

[Edit:] Yep should be able to tie objectives on the passes in nicely to that. Yeah I was aware of the logarithmic scales hiding a multitude of the sins, so it's great to see that awareness reflected in the non-log representation. Nose will be back to the grindstone a bit later.

Did I mention faces going blank and eyes glazing over?

At this stage the documentation is for our benefit. For many others the proof is in the pudding, and how we get there of little importance to them...

I don;t queston the importance of documentation.
It's just that you usuallt manage to talk at a level everybody else (including me) finds hard to understand. You ain't talkinf to fellow engeneers.
Try the 'explain to an intelligent 16 year old' approach. Else you end up explaining everything twice.

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

Yes, but "our benefit" includes the wider benefit of getting code changes accepted into the master BOINC codebase. Hand-waving demonstrations of "but look - it works" are simply descending to the level of the lowest common denominator. Recording and documenting the steps which led to the conclusions being implemented in code will help - even if only by giving us the warm fuzzies that come from occupying the moral high ground.

I've started documenting the wingmates who co-validate my 'high outlier' credit scores, but no pattern has emerged yet.

Only what you can explain you have understood properly.

Patterns in chaos? Yes there are some, but you could as easily try to fimd to identical snowflakes.
I'm intrigued by the high outliers too.

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

[Edit:] Yep should be able to tie objectives on the passes in nicely to that. Yeah I was aware of the logarithmic scales hiding a multitude of the sins, so it's great to see that awareness reflected in the non-log representation. Nose will be back to the grindstone a bit later.

Did I mention faces going blank and eyes glazing over?

At this stage the documentation is for our benefit. For many others the proof is in the pudding, and how we get there of little importance to them...

I don;t queston the importance of documentation.
It's just that you usuallt manage to talk at a level everybody else (including me) finds hard to understand. You ain't talkinf to fellow engeneers.
Try the 'explain to an intelligent 16 year old' approach. Else you end up explaining everything twice.

How is it you're managing to criticise a document you've never seen ? FWIW my elderly mother understands it so far, even having to ask 'what's a controller?'

I've got absolutely no intentions of condescending to you or anyone else on basic control systems theory within the experiment, though I can certainly point out some background reading.

If I must, I'll draw an analogy. When you drive your car you're a 'controller'. If you drive your car like Boinc drives estimates and credit, then according to that data you have no business being behind the wheel of a car at all.

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

[Edit:] Yep should be able to tie objectives on the passes in nicely to that. Yeah I was aware of the logarithmic scales hiding a multitude of the sins, so it's great to see that awareness reflected in the non-log representation. Nose will be back to the grindstone a bit later.

Did I mention faces going blank and eyes glazing over?

At this stage the documentation is for our benefit. For many others the proof is in the pudding, and how we get there of little importance to them...

Yes, but "our benefit" includes the wider benefit of getting code changes accepted into the master BOINC codebase. Hand-waving demonstrations of "but look - it works" are simply descending to the level of the lowest common denominator. Recording and documenting the steps which led to the conclusions being implemented in code will help - even if only by giving us the warm fuzzies that come from occupying the moral high ground.

I've started documenting the wingmates who co-validate my 'high outlier' credit scores, but no pattern has emerged yet.

Thanks :) yes it could be simply artefacts of the instabilities (and so the repeated mentions of chaos). It'll be a good marker to see if we squished all the gremlins out.

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

I've started documenting the wingmates who co-validate my 'high outlier' credit scores, but no pattern has emerged yet.

Validated with different app versions, like x86 on one and x64 on another?

Been running a number of CPU hosts on and off for months, mostly Arm, before the upgrade the best app, ie Neon app was only sent to my Arm hosts unless I aborted tasks to drive the Max tasks per day down low enough,
(My 2012 HTC One S and the 1.43 Neon app only produced validate errors, and the scheduler wouldn't send the 1.43 VFP app unless I did that, it completed 5 of those O.K),
the 1.44 Neon app is good through, and has completed over 200 hundred now with hardly a problem, no more VFP tasks have been sent.
The two Parallellas were only doing Neon tasks before hand, afterwards they started picking up non Neon tasks, they are at 11 and 10 validations so far for non Neon, and 21 and 23 for Neon,
The 2012 Nexus 7 had done only Neon tasks beforehand, afterwards it's picked up VFP tasks, done 8 of those against 37 of Neon, the VFP app is about half the speed of the Neon app,
The C2D T8100 Linux x64 host before hand only picked up x64 BRP tasks, it's completed 271 so far, SSE2 x86 tasks have never been sent,

On the HD7770 it picked up 1.34 windows_x86_64 (BRP4G-opencl-ati) and 1.34 windows_intelx86 (BRP4G-opencl-ati) work, these are different apps, with different file sizes, the x86_x64 app had some validations from beforehand,
afterwards they were failing with max time exceeded errors for a few days, the x86 work got sent when the x64 max tasks per day got too low,
since the x64 tasks got a reasonable speed estimate their tasks complete O.K, x86 tasks haven't been sent again, I have no idea which app is fastest,
looks as if there are scheduler differences between sending CPU and GPU apps, I would have expected some x86 work to be sent.

It's similar with the Perseus Arm Survey, I've had work from 1.39 windows_x86_64 (BRP5-opencl-ati) and 1.39 windows_intelx86 (BRP5-opencl-ati), the x64 app has some validations, the x86 none,
while you can't tell the difference from the tasks page, Boinc Manager shows different duration estimates for the two, 33secs for x64, 20secs for x86, stderr.txt doesn't seem to tell them apart.

CreditNew seems to use different calculations depending whether an app is above or below a sample level, could it be that one app version is above the sample level and the other isn't?

Claggy

...
CreditNew seems to use different calculations depending whether an app is above or below a sample level, could it be that one app version is above the sample level and the other isn't?...

Yes there are two levels above initial, being numbers of tasks validated. For host_scale (host app version) it's >10 validations, which should then switch from whetstone*elapsed related to PFC 'averages' (scaling kicks in). For the global scaling of app versions, it's >100 validations total from all hosts, which is the cross scaling point between all app versions. Both would make steps.

He's put some weights on those averages (to attempt) to reduce their (noisy) effects. That's the bit that's the same as a chip in a 5$ CD Player & doesn't work so well stability wise.

Late coming hosts (there's been >100 by others, <=10 locally) to the app *could* be getting their whetstone*elapsed (peak_flops*elapsed for the GPU case) value global scaled on top, that'd where it gets all spaghetified and the noise buries a lot.

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

Validated with different app versions, like x86 on one and x64 on another?

Claggy

Nice thought, and worth checking. But 11 of my 15 'high scores' have been for Windows / NVidia, and every one of them has been for the same (BRP4G-cuda32-nv301) application that I'm running.

Mind you, five of those 11 Windows 'high scores' have been against host 8733: scanning that machine's task list by eyeball only, one does seem to have a significantly higher chance of hitting the jackpot if paired with that host. Can anybody spot anything special in the published metrics for that host? I can't.

Validated with different app versions, like x86 on one and x64 on another?

Claggy

Nice thought, and worth checking. But 11 of my 15 'high scores' have been for Windows / NVidia, and every one of them has been for the same (BRP4G-cuda32-nv301) application that I'm running.

Mind you, five of those 11 Windows 'high scores' have been against host 8733: scanning that machine's task list by eyeball only, one does seem to have a significantly higher chance of hitting the jackpot if paired with that host. Can anybody spot anything special in the published metrics for that host? I can't.

Having a squizz now.

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage