mersenneforum.org  

Go Back   mersenneforum.org > Great Internet Mersenne Prime Search > Hardware

Reply
 
Thread Tools
Old 2021-04-08, 00:00   #1
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

47·79 Posts
Default Any Infiniband Gurus Available?

I have acquired two cards and a cable and am hoping to connect two Z620s for LA work. I have installed one card in one machine, but nothing in the other, yet. So far, I've tried several web pages that are supposed to walk me though the process, but things don't succeed for me the way the pages succeed.

One example, I installed rmda-core, and was supposed to run systemctl start rmda.service, but the OS says there is no rmda.service. opensm installed and its start service returned as expected.

The cards don't appear to have a name, but their Model# is HSTNS-BN80.

The machines are Z620 dual Xeons running Ubuntu 20.04.

A forum search for "infiniband" turned up three pages of threads, but I couldn't detect any helpful possibilities via the titles.

Any assistance would be appreciated.
EdH is offline   Reply With Quote
Old 2021-04-08, 00:25   #2
VBCurtis
 
VBCurtis's Avatar
 
"Curtis"
Feb 2005
Riverside, CA

477510 Posts
Default

I am very interested in doing this exact-same thing, if Ed gets his setup working. Two cards and a cable are fairly cheap used, and connecting just two machines means no need for a switch.

I'm told (well, I read on a website like the ones Ed found) that even with 2-port cards, one cannot connect three machines without a switch. I'd like to know if that is true!
VBCurtis is offline   Reply With Quote
Old 2021-04-08, 07:00   #3
frmky
 
frmky's Avatar
 
Jul 2003
So Cal

3·17·41 Posts
Default

I haven't tried this in quite a while, but when I built my own small clusters I used Rocks. That configures IB out of the box, as well as MPI supporting IB, a resource manager, and a job scheduler.
frmky is online now   Reply With Quote
Old 2021-04-08, 11:19   #4
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

47·79 Posts
Default

Quote:
Originally Posted by frmky View Post
I haven't tried this in quite a while, but when I built my own small clusters I used Rocks. That configures IB out of the box, as well as MPI supporting IB, a resource manager, and a job scheduler.
Thanks. This looks like it requires CentOS, or am I mistaken? It also looks like nothing has been done with it since 2017. I'll study it a bit more.
EdH is offline   Reply With Quote
Old 2021-04-09, 00:08   #5
frmky
 
frmky's Avatar
 
Jul 2003
So Cal

82B16 Posts
Default

Yes, in my experience most cluster systems are built on a Red Hat-based distro. Although it looks like Rocks hasn't been updated in a while, and the community appears to be moving to OpenHPC.

https://openhpc.community/
https://github.com/XSEDE/CRI_XCBC/tree/master/doc
frmky is online now   Reply With Quote
Old 2021-04-09, 02:35   #6
EdH
 
EdH's Avatar
 
"Ed Hall"
Dec 2009
Adirondack Mtns

371310 Posts
Default

Quote:
Originally Posted by frmky View Post
Yes, in my experience most cluster systems are built on a Red Hat-based distro. Although it looks like Rocks hasn't been updated in a while, and the community appears to be moving to OpenHPC.

https://openhpc.community/
https://github.com/XSEDE/CRI_XCBC/tree/master/doc
Thanks! I'll look these over. I currently have an openmpi cluster of various machines that's running over ethernet. I was kind of hoping I could just get the communication between two of the machines to link with a couple Infiniband cards, so I could get enough bandwidth to run Msieve LA a bit faster. I used to run it with Gigabit until the machines got so fast Gigabit couldn't handle it.
EdH is offline   Reply With Quote
Reply

Thread Tools


Similar Threads
Thread Thread Starter Forum Replies Last Post
Bootable Prime95 Linux - gurus needed tantryl Software 55 2008-06-09 00:30
Factoring For Non-RSA-Gurus koal Puzzles 5 2003-06-27 08:11

All times are UTC. The time now is 20:35.

Sun May 9 20:35:34 UTC 2021 up 31 days, 15:16, 1 user, load averages: 2.71, 3.30, 3.69

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.