Intel Xeon PHI?

[paulunderwood]

Originally Posted by ewmayer

Installed gcc (it grabs 8.3.1) and gdb ... ssh is clearly already working, and python3 also there. One yum no-package-found does concern me, though:

No match for argument: libgmp-dev

Edit /etc/dnf/dnf.conf line to "skip_if_unavailable=true".

Run dnf distro-sync

Run dnf search gmp and you will see that it is called something else.

Run dnf install gmp-devel.x86_64

(I turned on some repositories through the software gui.)

In old money that would be yum gmp-devel.86_64

[ewmayer]

@Paul: 'yum list available >& pkgs.txt' gives a 5285-line file, a bzip2 of which is attached. Of especial interest to me - you already noted the first one:

gmp-devel.x86_64
lm_sensors.x86_64

Repeating the above but replacing yum->dnf and pkgs->pkg2 shows the yum output to differ via just 3 extra lines over dnf:

CentOS-8 - AppStream                            4.1 MB/s | 5.8 MB     00:0
CentOS-8 - Base                                 2.9 MB/s | 2.2 MB     00:00
CentOS-8 - Extras                                21 kB/s | 8.6 kB     00:00

With the system under heavy load (Mlucas avx-512 build1-thread on each of 64 of the 68 cores) - more on that below - 'sensors' gives the following, where I am 99% sure the 'Core' lines are bogus - 74 cores mismatches the actual core count of 68 and I have bolded the actual CPU output:

[ewmayer@localhost ~]$ sensors
i350bb-pci-0400
Adapter: PCI adapter
loc1:         +49.0°C  (high = +120.0°C, crit = +110.0°C)

coretemp-isa-0000
Adapter: ISA adapter
Package id 0:  +53.0°C  (high = +80.0°C, crit = +90.0°C)
Core 0:        +37.0°C  (high = +80.0°C, crit = +90.0°C)
...
Core 73:       +35.0°C  (high = +80.0°C, crit = +90.0°C)

On my Intel NUC minis I use the combination 'sensors|grep Package', here are sample outputs for my old 1"-high Broadwell NUC (Mlucas avx2 build) and my newish 2"-high dual-core 8i3CYSM NUC (Mlucas avx512 build), both 2-physical-core and running 4-threaded:

Broadwell: Package id 0: +75.0°C (high = +105.0°C, crit = +105.0°C)
8i3CYSM: Package id 0: +69.0°C (high = +105.0°C, crit = +105.0°C)

So if the 53.0C for the KNL is to be believed - and the fact that a similar run using 'only' 32 cores gives a cooler 44.0C indicates so - that water cooling is working very well indeed.

Been doing Mlucas build timings most of the day - self-tests at the big FFT lengths from 64-512M are rather time-consuming. The performance appears more or less the same as that of the AWOL crowdfunded GIMPS KNL for same-build-and-#cores-used runs, though the added 4 cores of the Hydra workstation provide a little bonus headroom for added tasks.

[kriesel]

Windows 10 & prime95:
Cpuid HWMonitor provides no core temp output
Cpuid HWInfo crashes after displaying some cpu loadings
CoreTemp https://www.alcpu.com/CoreTemp/ runs out of screen height, displaying 28 of 68 cores, with some interesting loading figures. The temps are comparable to Ernst's. Note the cooling is effective enough that with 32% cpu loading (25% would be one saturating task per physical core, due to the way Windows handles hyperthreading), a degree of turbo clocking still occurs. Nominal is 1.4, full turbo is 1.5 Ghz, I've seen 1.42 and 1.44 with full prime95 load.
But when minimized, CoreTemp goes to the System Tray. Bringing up the System Tray shows a real time temp for each of the 68 cores there. Highest I've seen is 56C, and the house ambient here may be higher than Ernst's apartment.
Open Hardware Monitor https://openhardwaremonitor.org/downloads/ displays loads for the first 60 cores but no temps.

[EdH]

Glad to see the long drawn out process seeming to come to a point where it can be useful instead of nerve wracking. But the question is formed in my thoughts as to whether a mere mortal who knows less than he thinks and possesses ancient hardware should acquire a less capable (5110P) version of such to play with. . .

[kriesel]

Originally Posted by EdH

Glad to see the long drawn out process seeming to come to a point where it can be useful instead of nerve wracking. But the question is formed in my thoughts as to whether a mere mortal who knows less than he thinks and possesses ancient hardware should acquire a less capable (5110P) version of such to play with. . .

P means passive cooling; it's designed to go in a server package that pushes air through it lengthwise. So be prepared to come up with some sort of custom HVAC solution. (3d print an adapter?) Or hold out for the rare occasional A such as 7120A which has an integral fan. Then grapple with MPSS to run it as a coprocessor in a hosting system.

[EdH]

Originally Posted by kriesel

P means passive cooling; it's designed to go in a server package that pushes air through it lengthwise. So be prepared to come up with some sort of custom HVAC solution. (3d print an adapter?) Or hold out for the rare occasional A such as 7120A which has an integral fan. Then grapple with MPSS to run it as a coprocessor in a hosting system.

Thanks! I have been following the whole thread, although my memory of the earlier posts is missing, and was aware of the passive cooling but thanks for mentioning it in case I hadn't caught that. I'm more concerned with whether any of my current hardware would actually allow it to function. Is there a BIOS feature needed, that I don't have? Is the PCIe connector something I may not have? Only a couple of my machines have the 6-pin power connector available and one of those is an old Core2 Duo server system. All of my machines are currently running Ubuntu or Debian. Is the CentOS needed on the host machine or somewhere (VM?) for the PHI? The 5110P is currently available for ~$50 from Ebay, but would it work with any of my current machines?

[kriesel]

Originally Posted by EdH

Is there a BIOS feature needed, that I don't have? Is the PCIe connector something I may not have? Only a couple of my machines have the 6-pin power connector available and one of those is an old Core2 Duo server system. All of my machines are currently running Ubuntu or Debian. Is the CentOS needed on the host machine or somewhere (VM?) for the PHI? The 5110P is currently available for ~$50 from Ebay, but would it work with any of my current machines?

7120A needs:
PCIe socket (I think any flavor)
2 slots width
power connectors for 300W TDP: 75W PCIe slot + 75W 6-pin connector +150W 8pin connector
PCIe support for >4GB addressing in the system BIOS

5110P needs cooling air flow provided by the system it's installed in (think high pressure blower, not ordinary fan) or a custom connection to a high pressure blower
PCIe socket
2 slots width
power connector for 225W TDP (75W from PCIe, other 150W from 8-pin connector)
PCIe support for >4GB addressing in the system BIOS

MPSS is the software that runs on the host system to talk to the embedded linux on the coprocessor card. Versions are available for Linux and for Windows hosts.

Both of those are Knights Corner, not supporting AVX512.

Knights Landing as the basis of a system apparently needs either RHEL or Centos, or a current build of Windows 10. No MPSS needed. It's the main (only) processor in the system in that case. There were also coprocessor cards made with Knights Landing; 72xx, very hard to find.
https://en.wikipedia.org/wiki/Xeon_Phi

[EdH]

Thanks! This helps a lot. The only system I have that sounds even remotely like it would work is an ancient ASUS? Core2 Duo and I'd be if surprised the BIOS has what is needed. I don't think anything else I have has the power connectors.

Oh, well! Saves ne some money for now. . .

[kriesel]

Originally Posted by EdH

Thanks! This helps a lot. The only system I have that sounds even remotely like it would work is an ancient ASUS? Core2 Duo and I'd be if surprised the BIOS has what is needed. I don't think anything else I have has the power connectors.

Oh, well! Saves ne some money for now. . .

You've been wanting to upgrade your power supply to a platinum rated with extra power to operate at top efficiency, anyway, right?

[EdH]

Originally Posted by kriesel

You've been wanting to upgrade your power supply to a platinum rated with extra power to operate at top efficiency, anyway, right?

I guess I'd have to. The one system I thought "might" be capable has a 650 BFG supply, but looking it over it has two 6-pin and one 8-pin plus the main and drive connectors. There does not appear to be a 4-pin for the motherboard. Instead the motherboard is using the 8-pin, which means it would be unavailable for the PHI. On the cooling side, that particular case has large fans all over it.

[ewmayer]

Originally Posted by EdH

I guess I'd have to. The one system I thought "might" be capable has a 650 BFG supply, but looking it over it has two 6-pin and one 8-pin plus the main and drive connectors. There does not appear to be a 4-pin for the motherboard. Instead the motherboard is using the 8-pin, which means it would be unavailable for the PHI. On the cooling side, that particular case has large fans all over it.

My old Haswell system has an old non-modular PSU with just a single 8-pin PCIe ... I manage to host 2 Radeon VIIs in it by ganging together the 2 6-pin PCI connectors into another 8-pin, then using splitter cables to split each 8-pin PCIe in two. That works wattage-wise because I run the R7s underclocked, each drawing ~200W.

Back during the wild-n-cracy months of summer this year I stretched my open-testframe build, described over in the "Radeon for $500 on eBay" thread, to host a 4th R7 by "Frankencabling" a pair of SATA cables with 4 STA plugs along each: 2 SATAs into a 6-pin PCIe, 2 6-pin PCIe into a single 8-pin PCIe. The min-case Hydra KNL system Ken and I each bought has a cheapie PSU with size & shape roughly that of a carton of cigarettes with only SATA outputs to spare so the Frankencable trick might be an option to host a GPU, but the shape of the PSU precludes a large-format GPU, so that would need a new PSU in any event.

But I suggest if your decide to invest in any Intel 512-bit-vector-capable CPU, it be one running the AVX-512 instruction set that premiered on the KNL, not the older Knights Corner or whatever.