Deprecated: Function get_magic_quotes_gpc() is deprecated in /srv/BOINC/live-webcode/html/inc/util.inc on line 640
Project server code update

WARNING: This website is obsolete! Please follow this link to get to the new Albert@Home website!

Project server code update

Message boards : News : Project server code update
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 8 · 9 · 10 · 11 · 12 · 13 · 14 . . . 17 · Next

AuthorMessage
Richard Haselgrove

Send message
Joined: 10 Dec 05
Posts: 450
Credit: 5,409,572
RAC: 0
Message 113090 - Posted: 18 Jun 2014, 23:21:01 UTC - in response to Message 113089.  

OK, we have a working theory for the pseudo-random outliers, subject to wider checking tomorrow (after shut-eye) across my whole gamut of wingmates.

Next up: the gradual upward drift in credit across the population as a whole?


Will have to stew on that one for sure :). Perhaps look at the deadlines assigned to older CPUs (I presume far from expired ?). Since their Boinc Whetstones would be closer to their actual throughput, they should gradually roll in late to the party and cause a levelling. Fast, early, hosts by contrast drive things down by virtue of saturating the early statistics. The efefcts of that may be tempered here, by the app being resitricted to SSE2 (? like seti's astropulse), so the extreme latecomers would be SSE amd FPU (if available, e.g. Pentium 1/II. android with no vfp or NEON)

I've started on the process of retrieving 'time of validation' (later of [self, wingmate]), and wingmate data for non-lottery-winners. Will take a while, to guard against RSI.
ID: 113090 · Report as offensive     Reply Quote
Eyrie

Send message
Joined: 20 Feb 14
Posts: 47
Credit: 2,410
RAC: 0
Message 113091 - Posted: 19 Jun 2014, 8:44:53 UTC

A few points.

a) I think you corroborated my conclusion/gut feeling/code feeling that high APR/throughput leads to less credit (subject to noise)

b) The upwards drift is on the BRP4G app - we've not looked at CPU for some time. Don't know if Richard feels like more database wringing.

c) ... strike that - pour oil on the water not fan the flames.
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
ID: 113091 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 10 Dec 05
Posts: 450
Credit: 5,409,572
RAC: 0
Message 113092 - Posted: 19 Jun 2014, 8:49:25 UTC - in response to Message 113091.  

b) The upwards drift is on the BRP4G app - we've not looked at CPU for some time. Don't know if Richard feels like more database wringing.

I'm keeping an eye on my CPU runs as the gaps slowly fill in, but not generating any new data. Haven't spotted any sign of a drift.

But I'm coming up to a run of resends on BRP4G, so new trend data will fill in quickly there.
ID: 113092 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 10 Dec 05
Posts: 450
Credit: 5,409,572
RAC: 0
Message 113093 - Posted: 19 Jun 2014, 9:37:55 UTC

I've just validated 1,852.58 against a TITAN, and 2,315.85 against a GeForce GTX 680. On the basis of last night's discussion, I should the junior partner in both cases.

It goes on going up. Graphs later.
ID: 113093 · Report as offensive     Reply Quote
jason_gee

Send message
Joined: 4 Jun 14
Posts: 109
Credit: 1,043,639
RAC: 0
Message 113094 - Posted: 19 Jun 2014, 11:21:53 UTC - in response to Message 113092.  

b) The upwards drift is on the BRP4G app - we've not looked at CPU for some time. Don't know if Richard feels like more database wringing.

I'm keeping an eye on my CPU runs as the gaps slowly fill in, but not generating any new data. Haven't spotted any sign of a drift.

But I'm coming up to a run of resends on BRP4G, so new trend data will fill in quickly there.


With the CPU case my feeling is that if there was any drift, it would be over a much longer period. The actual throughputs are closer to estimated peaks. Since the estimated peak throughputs are incorrectly low figures though, the actual throughputs make for a higher ratio, levelling the overall scaling downward. Once the latest/slowest comers roll in (highest pfc's... highest APR/peak_flops), if not already, then those should indeed be relatively stable (even if scaled to the wrong value)
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
ID: 113094 · Report as offensive     Reply Quote
jason_gee

Send message
Joined: 4 Jun 14
Posts: 109
Credit: 1,043,639
RAC: 0
Message 113095 - Posted: 19 Jun 2014, 11:27:44 UTC - in response to Message 113093.  

I've just validated 1,852.58 against a TITAN, and 2,315.85 against a GeForce GTX 680. On the basis of last night's discussion, I should the junior partner in both cases.

It goes on going up. Graphs later.


Guessing from Zombie's TITAN APR, might start to show signs of levelling at around 5-10x expected.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
ID: 113095 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 10 Dec 05
Posts: 450
Credit: 5,409,572
RAC: 0
Message 113096 - Posted: 19 Jun 2014, 11:48:59 UTC - in response to Message 113095.  

Graphs later. (much later. I have to keep giving the monitor a rest)

Guessing from Zombie's TITAN APR, might start to show signs of levelling at around 5-10x expected.

In the meantime, try these two.

WU 620035
WU 606522

Same wingmate, validated within 80 minutes of each other. 450 (20%) difference in credit.

What's an Oland, anyway? Ah - it's slowed down. Maybe running 2-up now?
ID: 113096 · Report as offensive     Reply Quote
jason_gee

Send message
Joined: 4 Jun 14
Posts: 109
Credit: 1,043,639
RAC: 0
Message 113097 - Posted: 19 Jun 2014, 14:10:24 UTC - in response to Message 113096.  

... Maybe running 2-up now?


Or the machine's being actually used :-O
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
ID: 113097 · Report as offensive     Reply Quote
Profile Holmis

Send message
Joined: 4 Jan 05
Posts: 104
Credit: 2,104,736
RAC: 0
Message 113098 - Posted: 19 Jun 2014, 14:15:35 UTC

Here's some graphs from my validated tasks and to keep the post shorter I'll post links to the pictures.

BRP4X64 - A bit unstable, varies around 50 credits/task ±5 credits and with a few outliers.

BRP4G - Clear upward trend with no sign of coming back down. A few of the high outliers are late validations.

S6CasA (CPU only) - Not as unstable as BRPx64 but bigger difference between high and low, 270 - 330 credits/task. Low number of completed tasks so far.

BRP5 iGPU - The start of an upward trend with only 10 valid tasks so far.

BRP5 Nvidia - Big difference in high and low (20,59 - 8091,06) credits/task. Few completed tasks but might also be starting an upward trend.

And as before here's a link to the Excel document with all the data and graphs.
ID: 113098 · Report as offensive     Reply Quote
jason_gee

Send message
Joined: 4 Jun 14
Posts: 109
Credit: 1,043,639
RAC: 0
Message 113099 - Posted: 19 Jun 2014, 14:39:23 UTC - in response to Message 113098.  

Thanks! The Credit graphs have me giggling :)

What I might do a bit later is have a look at the data and see what happens with Credit vs Runtime.
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
ID: 113099 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 10 Dec 05
Posts: 450
Credit: 5,409,572
RAC: 0
Message 113100 - Posted: 19 Jun 2014, 15:49:05 UTC
Last modified: 19 Jun 2014, 16:03:48 UTC

OK, here's another graph for the mix.

Just my own host, linear scale. The green markers are when the actual validation/credit award was made (or, to be pedantic, when the validating task was reported). That should be a more honest trendline - if you can see one.



For clarification - if I report first, you'll see first a red mark (to the left) for my reporting time - that's what we've been using up until now. But it'll be followed by a green mark, some time later, when the wingmate trails in.

If I report second, validation takes place near-enough instantly, and the green mark will superimpose over the red mark - you'll only see green.
ID: 113100 · Report as offensive     Reply Quote
jason_gee

Send message
Joined: 4 Jun 14
Posts: 109
Credit: 1,043,639
RAC: 0
Message 113101 - Posted: 19 Jun 2014, 16:23:15 UTC - in response to Message 113100.  
Last modified: 19 Jun 2014, 16:24:14 UTC

Yes,
Looking through homlis' BRP4G I see similar trends. Elapsed per credit seems to be more or less levelling there (with a lot of noise). There appear to be 3 main populations early on, and two later as new hosts converge. New hosts & apps come along all the time, so understanding the credit award as average, low or jackpot regime helped out.

The 3 populations being:
- new host/app validating with anything (including if self is new), assumes high APR to peak_flops ratio (high pfc, of 10% default), with both hosts being high tends toward lower credit, so the opposite of the generous intent as written in code comments. (depending if those comments meant credit as expected, or actually elapsed estimate room. which is not specified)
- validating with another low pfc, such as both running multiple tasks on the GPU, would yield 'middle' credit
- and low pfc to high pfc validation, the jackpot scenario.

Oh, and plotted elapsed/credit as a bubble graph in supplied sequence gives freaky fractal Boinc skeleton hand. Winning :D


On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
ID: 113101 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 10 Dec 05
Posts: 450
Credit: 5,409,572
RAC: 0
Message 113102 - Posted: 19 Jun 2014, 16:32:41 UTC

Probably time for a stats show, then.

		Jason	Holmis	Claggy	Zombie	ZombieM	Jacob	RH
Host:		11363	2267	9008	6490	6109	10320	5367
		GTX 780	GTX 660	GT 650M	TITAN	680MX	FX3800M	GTX 670

Credit for BRP4G, GPU								
Maximum		1791.03	1982.00	10952.0	2794.94	11847.5	10952.0	4137.85
Minimum		115.82	88.84	153.90	91.50	94.88	508.73	1355.49
Average		1003.97	895.37	4105.99	1105.00	2009.54	2541.91	2040.89
Median		1168.97	981.15	3140.36	1144.04	1664.94	1570.34	1832.12
Std Dev		576.67	570.61	3015.77	424.29	1790.58	2873.96	495.61
								
nSamples	47	65	52	401	196	21	118

Runtime (seconds)								
Maximum		5027.36	5088.99	11295.0	5383.71	23977.4	12149.5	4169.93
Minimum		3259.10	3294.83	8122.09	1908.26	1512.16	11515.5	4061.45
Average		3674.58	4558.79	8906.78	4191.69	4207.76	11652.6	4119.66
Median		3581.10	4818.30	8837.27	4224.43	4175.31	11617.4	4123.75
Std Dev		346.83	486.74	554.28	550.95	1947.62	131.33	20.85

Turnround (days)								
Maximum		6.09	3.91	2.75	3.44	2.94	7.49	0.87
Minimum		0.15	0.07	0.14	0.05	1.52	0.18	0.15
Average		1.66	1.83	0.84	2.09	2.33	2.14	0.61
Median		1.46	1.85	0.78	2.00	2.45	1.02	0.65
Std Dev		1.64	1.01	0.63	0.61	0.32	2.27	0.15
ID: 113102 · Report as offensive     Reply Quote
Eyrie

Send message
Joined: 20 Feb 14
Posts: 47
Credit: 2,410
RAC: 0
Message 113103 - Posted: 19 Jun 2014, 16:34:58 UTC

'bubble graph' what the hack does a bubble graph graph?
yes i see differently sized bubbles. What does the bubble size stand for?
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
ID: 113103 · Report as offensive     Reply Quote
jason_gee

Send message
Joined: 4 Jun 14
Posts: 109
Credit: 1,043,639
RAC: 0
Message 113104 - Posted: 19 Jun 2014, 16:37:05 UTC - in response to Message 113103.  
Last modified: 19 Jun 2014, 16:37:26 UTC

'bubble graph' what the hack does a bubble graph graph?
yes i see differently sized bubbles. What does the bubble size stand for?

seconds elapsed per credit, so smaller is better paying.

That one's less for technical value, more for the fractal art competition anyway ;)
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
ID: 113104 · Report as offensive     Reply Quote
jason_gee

Send message
Joined: 4 Jun 14
Posts: 109
Credit: 1,043,639
RAC: 0
Message 113105 - Posted: 19 Jun 2014, 16:40:37 UTC - in response to Message 113102.  

Probably time for a stats show, then.


What happens, to the std deviations in particular, if you filter out the on-ramping converge attempt period, of say 10-20 results worth ?
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
ID: 113105 · Report as offensive     Reply Quote
Eyrie

Send message
Joined: 20 Feb 14
Posts: 47
Credit: 2,410
RAC: 0
Message 113107 - Posted: 19 Jun 2014, 16:44:14 UTC - in response to Message 113104.  

'bubble graph' what the hack does a bubble graph graph?
yes i see differently sized bubbles. What does the bubble size stand for?

seconds elapsed per credit, so smaller is better paying.

That one's less for technical value, more for the fractal art competition anyway ;)

Oh you are plotting against n not against N .....

sorry my maths genes need a serious liedown now.
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
ID: 113107 · Report as offensive     Reply Quote
jason_gee

Send message
Joined: 4 Jun 14
Posts: 109
Credit: 1,043,639
RAC: 0
Message 113108 - Posted: 19 Jun 2014, 16:52:31 UTC - in response to Message 113107.  

'bubble graph' what the hack does a bubble graph graph?
yes i see differently sized bubbles. What does the bubble size stand for?

seconds elapsed per credit, so smaller is better paying.

That one's less for technical value, more for the fractal art competition anyway ;)

Oh you are plotting against n not against N .....

sorry my maths genes need a serious liedown now.


*points Boinc bony fractal finger* everyone's an art critic :P
On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage
ID: 113108 · Report as offensive     Reply Quote
Richard Haselgrove

Send message
Joined: 10 Dec 05
Posts: 450
Credit: 5,409,572
RAC: 0
Message 113109 - Posted: 19 Jun 2014, 16:58:59 UTC - in response to Message 113105.  

Probably time for a stats show, then.

What happens, to the std deviations in particular, if you filter out the on-ramping converge attempt period, of say 10-20 results worth ?

I think it would be difficult to define an on-ramp in this case. Zombie, in particular, has been crunching for ages - I'm not sure how come we started the whole new convergence with the server code upgrade (I'd have expected all the runtime averages to have been in play for a long time, just disguised by the project's fixed credit policy). As you'll have been seeing, there hasn't been much tweaking in this area of code since it was first deployed four years ago.

What I would like to say to David is that, with fixed-size workunits, and a runtime standard deviation of 20 seconds in 4,000, I expect credit with an SD of 5 in 1000.
ID: 113109 · Report as offensive     Reply Quote
Eyrie

Send message
Joined: 20 Feb 14
Posts: 47
Credit: 2,410
RAC: 0
Message 113110 - Posted: 19 Jun 2014, 17:07:41 UTC

sample size is a bit low for good statistics but with THAT graph you can;t expect a reasonable SD.
better not try plotting credit against runtime - at least not when you expect the linear regression to have any sensible R^2 value...
Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.
ID: 113110 · Report as offensive     Reply Quote
Previous · 1 . . . 8 · 9 · 10 · 11 · 12 · 13 · 14 . . . 17 · Next

Message boards : News : Project server code update



This material is based upon work supported by the National Science Foundation (NSF) under Grant PHY-0555655 and by the Max Planck Gesellschaft (MPG). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the investigators and do not necessarily reflect the views of the NSF or the MPG.

Copyright © 2024 Bruce Allen for the LIGO Scientific Collaboration