Very variable gpu duration
log in

Advanced search

Message boards : AMD : Very variable gpu duration

Author Message
[VENETO] boboviz
Send message
Joined: 16 Nov 16
Posts: 37
Credit: 1,140,113
RAC: 0
Message 10 - Posted: 16 Nov 2016, 20:35:47 UTC

I have some wus, from 4 seconds to 2 minutes on my Rx260X
Is it normal??

Vlad
Project administrator
Project developer
Project tester
Project scientist
Help desk expert
Send message
Joined: 26 Oct 16
Posts: 322
Credit: 103,382
RAC: 0
Message 15 - Posted: 17 Nov 2016, 0:12:53 UTC - in response to Message 10.
Last modified: 17 Nov 2016, 8:44:54 UTC

Yes, this is normal. For the same material, the computational time is proportional to the square of the crystallite size, so, the computations for 21 nm crystallite will take 12.25 times longer than for 6 nm crystallite. However, the atomic density also varies from one material to another. There are also other parameters that may affect the computational time.

See the correct answer in the next message.

Vlad
Project administrator
Project developer
Project tester
Project scientist
Help desk expert
Send message
Joined: 26 Oct 16
Posts: 322
Credit: 103,382
RAC: 0
Message 17 - Posted: 17 Nov 2016, 8:41:33 UTC - in response to Message 10.

Sorry, I wrote my answer at 3 am local time, and it is terribly wrong. The time, t, required to compute the powder diffraction pattern of a crystallite is proportional to the square of the number of atoms in it, N. N is proportional to the cube of the crystallite size, D. So, t ~ D6. So, the computation of powder diffraction pattern for 21 nm crystallite takes 1838 longer than the computation for 6 nm crystallite (if the atomic density is the same). However, for any crystallite, the app needs to compute the atomic ensemble, print the files, etc. That’s why the lowest computation time is always a second or so.
In the future I'll improve the scheduler, so the initial data for the small crystallites will be sent only to the CPUs.

Steve Hawker*
Send message
Joined: 11 Nov 16
Posts: 13
Credit: 44,799
RAC: 0
Message 18 - Posted: 17 Nov 2016, 14:36:13 UTC

i found that you need to reserve one whole CPU core otherwise it can take an age.

Tasks on my 7950 were taking 5 mins or so compared to less than 10 seconds on similar GPUs. Once i made sure it had a full core, times dropped to a few seconds.

Vlad
Project administrator
Project developer
Project tester
Project scientist
Help desk expert
Send message
Joined: 26 Oct 16
Posts: 322
Credit: 103,382
RAC: 0
Message 19 - Posted: 17 Nov 2016, 16:11:07 UTC - in response to Message 18.
Last modified: 17 Nov 2016, 16:11:37 UTC

i found that you need to reserve one whole CPU core otherwise it can take an age.

Tasks on my 7950 were taking 5 mins or so compared to less than 10 seconds on similar GPUs. Once i made sure it had a full core, times dropped to a few seconds.

Thank you, this is very valuable information! I’ve never tested this project in parallel with some CPU-intensive projects.

If I understand this correctly, setting the cpu_frac to 0.5 in the plan class configuration will guarantee that one CPU core will be reserved for the GPU app. I need to test this on my VM first.

Also, it is interesting, whether this is required only for the OpenCL version (due to runtime kernel compilation) or for the CUDA version too.

[VENETO] boboviz
Send message
Joined: 16 Nov 16
Posts: 37
Credit: 1,140,113
RAC: 0
Message 20 - Posted: 17 Nov 2016, 17:12:31 UTC - in response to Message 18.

i found that you need to reserve one whole CPU core otherwise it can take an age.
Tasks on my 7950 were taking 5 mins or so compared to less than 10 seconds on similar GPUs. Once i made sure it had a full core, times dropped to a few seconds.


Other "gpu projects" need a dedicated cpu core.
App_info.xml solves the problem

[VENETO] boboviz
Send message
Joined: 16 Nov 16
Posts: 37
Credit: 1,140,113
RAC: 0
Message 21 - Posted: 17 Nov 2016, 17:15:46 UTC - in response to Message 17.

In the future I'll improve the scheduler, so the initial data for the small crystallites will be sent only to the CPUs.


Good idea.
If you send too much little wus to gpu, you risks a DOS of scheduler because of the intense upload/download. Milkyway@Home, recently, has increased 5 times the data in gpu app to reduce the server load.

Vlad
Project administrator
Project developer
Project tester
Project scientist
Help desk expert
Send message
Joined: 26 Oct 16
Posts: 322
Credit: 103,382
RAC: 0
Message 22 - Posted: 17 Nov 2016, 17:39:16 UTC - in response to Message 21.


Good idea.
If you send too much little wus to gpu, you risks a DOS of scheduler because of the intense upload/download. Milkyway@Home, recently, has increased 5 times the data in gpu app to reduce the server load.

It seems that this feature is perfect for my case. I'll try to implement it.

Vlad
Project administrator
Project developer
Project tester
Project scientist
Help desk expert
Send message
Joined: 26 Oct 16
Posts: 322
Credit: 103,382
RAC: 0
Message 23 - Posted: 17 Nov 2016, 18:36:48 UTC - in response to Message 20.


Other "gpu projects" need a dedicated cpu core.
App_info.xml solves the problem

The parameters responsible for the reservation of CPU cores are avg_ncpus and max_ncpus. Is that right?
They can be specified both in app_info.xml on the client side and in plan_class_spec.xml on the server side.
So, if I add:
<avg_ncpus>1.</avg_ncpus> <max_ncpus>1</max_ncpus>

to all GPU plan classes in plan_class_spec.xml, this should solve the problem for all volunteers in theory. Am I right?

Vlad
Project administrator
Project developer
Project tester
Project scientist
Help desk expert
Send message
Joined: 26 Oct 16
Posts: 322
Credit: 103,382
RAC: 0
Message 37 - Posted: 19 Nov 2016, 18:57:19 UTC - in response to Message 18.

i found that you need to reserve one whole CPU core otherwise it can take an age.

I've updated the GPU plan classes by adding:
<avg_ncpus>1.</avg_ncpus> <max_ncpus>1</max_ncpus>

Is the problem solved?

Message boards : AMD : Very variable gpu duration


Main page · Your account · Message boards


© 2021 Vladislav Neverov (NRC 'Kurchatov institute'), Nikolay Khrapov (Institute for Information Transmission Problems of RAS)