That's where it depends on how they hooked in those *4G and *5G aggregates of 16 tasks. If the estimate is standalone, then there is no visible reason it should give us 3 second estimates for hour long tasks. If it is hooked in via a multiple of the CPU apps PFC_Scale &/or CPU app estimate, then that would explain it.

I think that one is for Bernd and/or Oliver to answer. Or find the code for it...

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

That's where it depends on how they hooked in those *4G and *5G aggregates of 16 tasks. If the estimate is standalone, then there is no visible reason it should give us 3 second estimates for hour long tasks. If it is hooked in via a multiple of the CPU apps PFC_Scale &/or CPU app estimate, then that would explain it.

I think that one is for Bernd and/or Oliver to answer. Or find the code for it...

That's what I asked for the customisations for. In either case, It's still broke ;)

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

JASON, CPU and GPU have different APPS here!

there is a BRP app and a BRPG app. thta's like MB and AP on SETI...

or are you saying pfc for MB and AP are the same? :P

No, and don't yell at me please ;)

I'm female. If you don't listen I yell. At which point the voice becomes so highpitched, males go into automatic 'isn't she cute when she is upset' ignore mode :P Aren't sterotypes great?

I am saying that the BRP4G and 5G estimates come from CPU app estimates multiplied by 16 ---> different app & hardware, but estimates are linked.

that might be an explanation for that unreasonable extra scaling we are seeing, indeed.

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

In either case, It's still broke ;)

If It is broke, It needs to either work, win the lottery or find somebody that will keep It [I shudder to think what services It might supply in return].

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

That's where it depends on how they hooked in those *4G and *5G aggregates of 16 tasks. If the estimate is standalone, then there is no visible reason it should give us 3 second estimates for hour long tasks. If it is hooked in via a multiple of the CPU apps PFC_Scale &/or CPU app estimate, then that would explain it.

The runtime estimate we see displayed in a BOINC client doesn't come - as an estimate - from the server. Instead, we get two different numbers from the server - job size and host speed - and the client does the math.

size --> rsc_fpops_est
speed --> <flops>

I have no doubt that Bernd, Oliver et al will have set the rsc_fpops_est for the GPU app at 16* the est for the CPU app. We can check that, because rsc_fpops_est is set by hand into the workunit template, and transmitted *unchanged* through the workunit generator (analog of SETI's 'splitter') and on to our computers. At this project - thankfully - all workunits of a given type will have the same rsc_fpops_est.

You saw 3-second estimates because the *speed* term in the formula was wrong. We need to track that down, but it's nothing to do with a task size estimate. And it corrected itself once there was a usable APR for the host, to substitute for the faulty initial estimate.

That's where it depends on how they hooked in those *4G and *5G aggregates of 16 tasks. If the estimate is standalone, then there is no visible reason it should give us 3 second estimates for hour long tasks. If it is hooked in via a multiple of the CPU apps PFC_Scale &/or CPU app estimate, then that would explain it.

The runtime estimate we see displayed in a BOINC client doesn't come - as an estimate - from the server. Instead, we get two different numbers from the server - job size and host speed - and the client does the math.

size --> rsc_fpops_est
speed --> <flops>

I have no doubt that Bernd, Oliver et al will have set the rsc_fpops_est for the GPU app at 16* the est for the CPU app. We can check that, because rsc_fpops_est is set by hand into the workunit template, and transmitted *unchanged* through the workunit generator (analog of SETI's 'splitter') and on to our computers. At this project - thankfully - all workunits of a given type will have the same rsc_fpops_est.

You saw 3-second estimates because the *speed* term in the formula was wrong. We need to track that down, but it's nothing to do with a task size estimate. And it corrected itself once there was a usable APR for the host, to substitute for the faulty initial estimate.

Actually at least for anon platform the code says est and bound are being scaled by *something* ... checking what, and where non anon-platform sets those, is requiring some backtracking.

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

That's where it depends on how they hooked in those *4G and *5G aggregates of 16 tasks. If the estimate is standalone, then there is no visible reason it should give us 3 second estimates for hour long tasks. If it is hooked in via a multiple of the CPU apps PFC_Scale &/or CPU app estimate, then that would explain it.

The runtime estimate we see displayed in a BOINC client doesn't come - as an estimate - from the server. Instead, we get two different numbers from the server - job size and host speed - and the client does the math.

size --> rsc_fpops_est
speed --> <flops>

I have no doubt that Bernd, Oliver et al will have set the rsc_fpops_est for the GPU app at 16* the est for the CPU app. We can check that, because rsc_fpops_est is set by hand into the workunit template, and transmitted *unchanged* through the workunit generator (analog of SETI's 'splitter') and on to our computers. At this project - thankfully - all workunits of a given type will have the same rsc_fpops_est.

You saw 3-second estimates because the *speed* term in the formula was wrong. We need to track that down, but it's nothing to do with a task size estimate. And it corrected itself once there was a usable APR for the host, to substitute for the faulty initial estimate.

Actually at least for anon platform the code says est and bound are being scaled by *something* ... checking what, and where non anon-platform sets those, is requiring some backtracking.

Completely agree for anon_plat. But we're going down the other fork here.

...

Actually at least for anon platform the code says est and bound are being scaled by *something* ... checking what, and where non anon-platform sets those, is requiring some backtracking.

Completely agree for anon_plat. But we're going down the other fork here.

Yep we need both... looking for where the scheduler calls add_result_to_reply(), which is located in sched_send.cpp . If the preceeding raw estimates come straight from the WU generator (aka splitter as you say), and never get a scalem then there are further mysteries to track down further into the GPU portion of the expedition.

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

For your info, my i7-2600K/HD7770 is now picking up Gamma-ray pulsar search #3 tasks, the initial CPU estimates look O.K at 4hrs 55mins, the ATI estimates are at 5 seconds.
(This application type has CPU, Nvidia, ATI and Intel apps across Windows, Mac and Linux (But no Intel app on Linux))

All Gamma-ray pulsar search #3 tasks for computer 8143

Claggy

Actually at least for anon platform the code says est and bound are being scaled by *something* ... checking what, and where non anon-platform sets those, is requiring some backtracking.

One of those somethings is APR, since APR cant be sent as <flops> under anon.

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

Actually at least for anon platform the code says est and bound are being scaled by *something* ... checking what, and where non anon-platform sets those, is requiring some backtracking.

Completely agree for anon_plat. But we're going down the other fork here.

Yep we need both... looking for where the scheduler calls add_result_to_reply(), which is located in sched_send.cpp . If the preceeding raw estimates come straight from the WU generator (aka splitter as you say), and never get a scalem then there are further mysteries to track down further into the GPU portion of the expedition.

Let's focus on the NON-anon_plat case for now. That's the one in widest use across the BOINC community.

We can see the process in action in the server logs in this project. Taking my latest for reference:

2014-06-17 15:50:45.2751 [PID=29665]    [version] looking for version of einsteinbinary_BRP4G
2014-06-17 15:50:45.2751 [PID=29665]    [version] Checking plan class 'BRP4G-opencl-ati'
2014-06-17 15:50:45.2751 [PID=29665]    [version] plan_class_spec: parsed project prefs setting 'gpu_util_brp' : true : 0.480000
2014-06-17 15:50:45.2751 [PID=29665]    [version] plan_class_spec: No AMD GPUs found
2014-06-17 15:50:45.2751 [PID=29665]    [version] [AV#721] app_plan() returned false
2014-06-17 15:50:45.2751 [PID=29665]    [version] Checking plan class 'BRP4G-cuda32'
2014-06-17 15:50:45.2751 [PID=29665]    [version] plan_class_spec: parsed project prefs setting 'gpu_util_brp' : true : 0.480000
2014-06-17 15:50:45.2752 [PID=29665]    [version] plan_class_spec: driver version required max: 29053, supplied: 33788
2014-06-17 15:50:45.2752 [PID=29665]    [version] [AV#723] app_plan() returned false
2014-06-17 15:50:45.2752 [PID=29665]    [version] Checking plan class 'BRP4G-cuda32-nv301'
2014-06-17 15:50:45.2752 [PID=29665]    [version] plan_class_spec: parsed project prefs setting 'gpu_util_brp' : true : 0.480000
2014-06-17 15:50:45.2752 [PID=29665]    [version] [AV#716] (BRP4G-cuda32-nv301) setting projected flops based on host elapsed time avg: 69.17G
2014-06-17 15:50:45.2752 [PID=29665]    [version] [AV#716] (BRP4G-cuda32-nv301) comparison pfc: 95.26G  et: 69.17G
2014-06-17 15:50:45.2752 [PID=29665]    [version] Best app version is now AV716 (70.26 GFLOP)
2014-06-17 15:50:45.2752 [PID=29665]    [version] Checking plan class 'BRP4G-opencl-ati'
2014-06-17 15:50:45.2752 [PID=29665]    [version] plan_class_spec: parsed project prefs setting 'gpu_util_brp' : true : 0.480000
2014-06-17 15:50:45.2752 [PID=29665]    [version] plan_class_spec: No AMD GPUs found
2014-06-17 15:50:45.2752 [PID=29665]    [version] [AV#720] app_plan() returned false
2014-06-17 15:50:45.2752 [PID=29665]    [version] [AV#716] (BRP4G-cuda32-nv301) setting projected flops based on host elapsed time avg: 69.17G
2014-06-17 15:50:45.2752 [PID=29665]    [version] [AV#716] (BRP4G-cuda32-nv301) comparison pfc: 95.26G  et: 69.17G
2014-06-17 15:50:45.2752 [PID=29665]    [version] Best version of app einsteinbinary_BRP4G is [AV#716] (69.17 GFLOPS)
2014-06-17 15:50:45.2753 [PID=29665]    [send] est delay 0, skipping deadline check
2014-06-17 15:50:45.2753 [PID=29665]    [version] get_app_version(): getting app version for WU#620860 (p2030.20131124.G176.58-00.38.S.b6s0g0.00000_1008) appid:29
2014-06-17 15:50:45.2753 [PID=29665]    [version] returning cached version: [AV#716]
2014-06-17 15:50:45.2753 [PID=29665]    [send] est delay 0, skipping deadline check
2014-06-17 15:50:45.2800 [PID=29665]    [send] Sending app_version einsteinbinary_BRP4G 2 133 BRP4G-cuda32-nv301; projected 69.17 GFLOPS
2014-06-17 15:50:45.2801 [PID=29665]    [send] est. duration for WU 620860: unscaled 4048.01 scaled 4050.90
2014-06-17 15:50:45.2801 [PID=29665]    [send] [HOST#5367] sending [RESULT#1493785 p2030.20131124.G176.58-00.38.S.b6s0g0.00000_1008_0] (est. dur. 4050.90s (1h07m30s90)) (max time 80960.18s (22h29m20s17))

In my case, the main estimation variable is

setting projected flops based on host elapsed time avg: 69.17G

I see something new in there, too:

comparison pfc: 95.26G et: 69.17G

Later on, the server does it's own version of the runtime estimate: that's purely to check that it's not sending more work than the host requested. There's a little more scaling at that stage, but to account for hosts which don't run 24/7. My correction is tiny, but Eyrie would see a big rescale for restricted BOINC availability.

For your info, my i7-2600K/HD7770 is now picking up Gamma-ray pulsar search #3 tasks, the initial CPU estimates look O.K at 4hrs 55mins, the ATI estimates are at 5 seconds.
(This application type has CPU, Nvidia, ATI and Intel apps across Windows, Mac and Linux (But no Intel app on Linux))

Claggy

whetstone, Flops and rsc_fpops_est for GPu and CPU?

edit: 'please' - sorry ::)

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

In my case, the main estimation variable is

setting projected flops based on host elapsed time avg: 69.17G

I see something new in there, too:

comparison pfc: 95.26G et: 69.17G

Later on, the server does it's own version of the runtime estimate: that's purely to check that it's not sending more work than the host requested. There's a little more scaling at that stage, but to account for hosts which don't run 24/7. My correction is tiny, but Eyrie would see a big rescale for restricted BOINC availability.

I've seen that annotation before, somewhere.
rr_sim I think - can you look at a sample please, to check local boinc log against server values?

---

Queen of Aliasses, wielder of the SETI rolling pin, Mistress of the red shoes, Guardian of the orange tree, Slayer of very small dragons.

In my case, the main estimation variable is

setting projected flops based on host elapsed time avg: 69.17G

I see something new in there, too:

comparison pfc: 95.26G et: 69.17G

Later on, the server does it's own version of the runtime estimate: that's purely to check that it's not sending more work than the host requested. There's a little more scaling at that stage, but to account for hosts which don't run 24/7. My correction is tiny, but Eyrie would see a big rescale for restricted BOINC availability.

I've seen that annotation before, somewhere.
rr_sim I think - can you look at a sample please, to check local boinc log against server values?

17/06/2014 18:08:02 | | [rr_sim] start: work_buf min 25920 additional 3456 total 29376 on_frac 0.999 active_frac 1.000
17/06/2014 18:08:02 | Albert@Home | [rr_sim] 0.00: p2030.20130202.G202.32-01.96.N.b1s0g0.00000_3552_3 finishes (17271.46G/69.12G)
17/06/2014 18:08:02 | Albert@Home | [rr_sim] 2808.68: p2030.20131124.G176.58-00.38.S.b5s0g0.00000_3024_2 finishes (240866.14G/69.12G)
17/06/2014 18:08:02 | Albert@Home | [rr_sim] 3551.29: p2030.20131124.G176.58-00.38.S.b5s0g0.00000_2912_2 finishes (280000.00G/69.12G)
17/06/2014 18:08:02 | Albert@Home | [rr_sim] 4293.89: p2030.20131124.G176.30-00.82.S.b4s0g0.00000_3040_2 finishes (280000.00G/69.12G)

The first two must be running tasks, partially completed. But the next two show the same 280 e12 I posted earlier as rsc_fpops_est here, divided by the familiar speed from APR.

For your info, my i7-2600K/HD7770 is now picking up Gamma-ray pulsar search #3 tasks, the initial CPU estimates look O.K at 4hrs 55mins, the ATI estimates are at 5 seconds.
(This application type has CPU, Nvidia, ATI and Intel apps across Windows, Mac and Linux (But no Intel app on Linux))

Claggy

whetstone, Flops and rsc_fpops_est for GPu and CPU?

edit: 'please' - sorry ::)

CPU p_fpops is 4514900817.923695

HD7770 peak_flops is 3584000000000.000000

flops for the CPU app_version of hsgamma_FGRP3 is 845960315.482654

flops for the ATI GPU app_version of hsgamma_FGRP3 is 2950327174499.708000

rsc_fpops_est is 15000000000000.000000, with rsc_fpops_bound at 300000000000000.000000

With an Gamma-ray pulsar search #3 only request I got:

https://albert.phys.uwm.edu/host_sched_logs/8/8143

2014-06-17 17:18:23.1994 [PID=2155 ] [send] CPU: req 8330.13 sec, 0.00 instances; est delay 0.00
2014-06-17 17:18:23.1995 [PID=2155 ] [send] AMD/ATI GPU: req 8692.21 sec, 0.00 instances; est delay 0.00
2014-06-17 17:18:23.1995 [PID=2155 ] [send] work_req_seconds: 8330.13 secs
2014-06-17 17:18:23.1995 [PID=2155 ] [send] available disk 95.78 GB, work_buf_min 95040
2014-06-17 17:18:23.1995 [PID=2155 ] [send] on_frac 0.923624 active_frac 0.985800 gpu_active_frac 0.984082
2014-06-17 17:18:23.1995 [PID=2155 ] [send] CPU features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 popcnt aes syscall nx lm vmx tm2 pbe
2014-06-17 17:18:23.3103 [PID=2155 ] [mixed] sending locality work first


2014-06-17 17:18:23.3223 [PID=2155 ] [version] get_app_version(): getting app version for WU#604131 (LATeah0109C_32.0_0_-1.48e-10) appid:30
2014-06-17 17:18:23.3223 [PID=2155 ] [version] looking for version of hsgamma_FGRP3
2014-06-17 17:18:23.3224 [PID=2155 ] [version] Checking plan class 'FGRPopencl-ati'
2014-06-17 17:18:23.3234 [PID=2155 ] [version] reading plan classes from file '/BOINC/projects/AlbertAtHome/plan_class_spec.xml'
2014-06-17 17:18:23.3234 [PID=2155 ] [version] plan_class_spec: parsed project prefs setting 'gpu_util_fgrp' : true : 1.000000
2014-06-17 17:18:23.3234 [PID=2155 ] [version] [AV#911] (FGRPopencl-ati) adjusting projected flops based on PFC avg: 2950.33G
2014-06-17 17:18:23.3234 [PID=2155 ] [version] Best app version is now AV911 (85.84 GFLOP)
2014-06-17 17:18:23.3235 [PID=2155 ] [version] Checking plan class 'FGRPopencl-intel_gpu'
2014-06-17 17:18:23.3235 [PID=2155 ] [version] plan_class_spec: parsed project prefs setting 'gpu_util_fgrp' : true : 1.000000
2014-06-17 17:18:23.3235 [PID=2155 ] [version] [version] No Intel GPUs found
2014-06-17 17:18:23.3235 [PID=2155 ] [version] [AV#912] app_plan() returned false
2014-06-17 17:18:23.3235 [PID=2155 ] [version] Checking plan class 'FGRPopencl-nvidia'
2014-06-17 17:18:23.3235 [PID=2155 ] [version] plan_class_spec: parsed project prefs setting 'gpu_util_fgrp' : true : 1.000000
2014-06-17 17:18:23.3235 [PID=2155 ] [version] plan_class_spec: No NVIDIA GPUs found
2014-06-17 17:18:23.3235 [PID=2155 ] [version] [AV#925] app_plan() returned false
2014-06-17 17:18:23.3235 [PID=2155 ] [version] [AV#911] (FGRPopencl-ati) adjusting projected flops based on PFC avg: 2950.33G
2014-06-17 17:18:23.3235 [PID=2155 ] [version] Best version of app hsgamma_FGRP3 is [AV#911] (2950.33 GFLOPS)
2014-06-17 17:18:23.3236 [PID=2155 ] [send] est delay 0, skipping deadline check
2014-06-17 17:18:23.3264 [PID=2155 ] [send] Sending app_version hsgamma_FGRP3 7 111 FGRPopencl-ati; projected 2950.33 GFLOPS
2014-06-17 17:18:23.3265 [PID=2155 ] [CRITICAL] No filename found in [WU#604131 LATeah0109C_32.0_0_-1.48e-10]
2014-06-17 17:18:23.3265 [PID=2155 ] [send] est. duration for WU 604131: unscaled 5.08 scaled 5.59
2014-06-17 17:18:23.3265 [PID=2155 ] [send] [HOST#8143] sending [RESULT#1450173 LATeah0109C_32.0_0_-1.48e-10_1] (est. dur. 5.59s (0h00m05s59)) (max time 101.68s (0h01m41s68))
2014-06-17 17:18:23.3291 [PID=2155 ] [locality] send_old_work(LATeah0109C_32.0_0_-1.48e-10_1) sent result created 344.0 hours ago [RESULT#1450173]
2014-06-17 17:18:23.3291 [PID=2155 ] [locality] Note: sent NON-LOCALITY result LATeah0109C_32.0_0_-1.48e-10_1
2014-06-17 17:18:23.3292 [PID=2155 ] [locality] send_results_for_file(h1_0997.00_S6Direct)
2014-06-17 17:18:23.3365 [PID=2155 ] [locality] in_send_results_for_file(h1_0997.00_S6Direct, 0) prev_result.id=1488887

Claggy

For your info, my i7-2600K/HD7770 is now picking up Gamma-ray pulsar search #3 tasks, the initial CPU estimates look O.K at 4hrs 55mins, the ATI estimates are at 5 seconds.
(This application type has CPU, Nvidia, ATI and Intel apps across Windows, Mac and Linux (But no Intel app on Linux))

Claggy

whetstone, Flops and rsc_fpops_est for GPu and CPU?

edit: 'please' - sorry ::)

CPU p_fpops is 4514900817.923695

HD7770 peak_flops is 3584000000000.000000

flops for the CPU app_version of hsgamma_FGRP3 is 845960315.482654

flops for the ATI GPU app_version of hsgamma_FGRP3 is 2950327174499.708000

rsc_fpops_est is 15000000000000.000000, with rsc_fpops_bound at 300000000000000.000000

Claggy

Unfortunately I missed the server log for a fetch - just got a 'report only' RPC instead. Could you grab a log if it does another work_fetch, please?

I've seen that annotation before, somewhere.
rr_sim I think - can you look at a sample please, to check local boinc log against server values?

yes, were were there the other day digging out where whetstone was hiding. sched_version.cpp, estimate_flops() functions. That one for non- anon, and another slightly different for anon. For non-anon, Before statistics are gathered it's Boinc Whetstone for CPU (incidentally SIMD aware oin Android but not x86), and some mystery guesstimate for GPUs

---

On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - C Babbage

I've seen that annotation before, somewhere.
rr_sim I think - can you look at a sample please, to check local boinc log against server values?

yes, were were there the other day digging out where whetstone was hiding. sched_version.cpp, estimate_flops() functions. That one for non- anon, and another slightly different for anon. For non-anon, Before statistics are gathered it's Boinc Whetstone for CPU (incidentally SIMD aware oin Android but not x86), and some mystery guesstimate for GPUs

Those mystery guesstimates for GPUs are one of the major quarries for our quest.

Claggy's ATI is running at 2.95 Teraflops, to put it in simpler numbers.

Unfortunately I missed the server log for a fetch - just got a 'report only' RPC instead. Could you grab a log if it does another work_fetch, please?

I did another request, and suspended network:

https://albert.phys.uwm.edu/host_sched_logs/8/8143

Claggy