Skip to main content

MPICH2: mpdboot failure due to conflicting /etc/hosts entry

One single line in the /etc/hosts file on my Ubuntu Linux machine wasted my precious whole afternoon. I had some assumptions about the cause of this problem but I don't want to spend too much time investigating on what really caused the trouble. Anyway, here are the listings of what worked and what didn't:

/etc/hostname
ubuntu

mpd.hosts.cluster
master
worker-01

Bad /etc/hosts
127.0.0.1 localhost localhost.localdomain ubuntu
192.168.200.128 master
192.168.200.129 worker-01

Result
thitiv@ubuntu:~$ mpdboot -n 2 -f mpd.hosts.cluster
thitiv@master's password:
mpdboot_ubuntu (handle_mpd_output 359): failed to ping mpd on master; recvd output={}

Good /etc/hosts
127.0.0.1 localhost localhost.localdomain
192.168.200.128 ubuntu master
192.168.200.129 worker-01

Result
thitiv@ubuntu:~$ mpdboot -n 2 -f mpd.hosts.cluster
thitiv@worker-01's password:
thitiv@ubuntu:~$

At this point, with the /etc/hosts now fixed, the MPICH2 cluster could be booted up successfully.

I don't want to invest my time trying to understand this right now but I would really appreciate anyone explaining to me what went wrong.

Keywords: , , , , ,

Comments

wheatazogg said…
Good evening!

Usually, you have to make sure that all of the hosts are reachable from all of the machines on the cluster's internal network.

Is ubuntu an alias for one of the machines in the hosts file?

With your original /etc/hosts file:
127.0.0.1 localhost localhost.localdomain ubuntu
192.168.200.128 master
192.168.200.129 worker-01

If you share that hosts file on all of the machines, the name 'ubuntu' always refers to the machine you're on. That is, regardless of whether you're on master or worker-01, each machine things it's 'ubuntu' as well.

You can get serious identity issues when the MPDs try to ping each other if things are inconsistent. It helps to make sure that all of the nodes are listening on interfaces on the same subnet -- which you did by moving the name.

Matthew
8:24 AM
Dr. Thiti said…
Thank you very much for pointing out the loop-back issue, Matthew. I think this is the cause of the problem.
9:01 AM
Anonymous said…
Couple of questions, regarding your MPI blogs. Are you running Ubuntu on all the nodes? If yes, how did you setup rsh/rlogin on the nodes and the master? I couldn't get beyond setting up openssh-server, I need to supply password to every attempt -

ssh hostname date

requires password.
5:00 AM
Dr. Thiti said…
Hi Indy,

Yes, I was running Ubuntu on all nodes. I used apt-get install to install the OpenSSH package supplied with the Ubuntu distribution. I also compiled MPICH-2 from the latest source.

Unlike the MPICH-1 version that I used many years ago, I didn't have to setup an rlogin file for MPICH-2. I think MPICH-2 has switched from rsh/rlogin to SSH. So basically we should make sure that SSH itself works before we start installing MPICH-2.

I tried to simplify things a little bit by creating the same login names and passwords on all machines.

I would suggest you to consult an SSH how-to document and try to SSH between machines before you proceed with the MPICH-2 installation.

Thiti.
10:23 AM
Kanibal said…
Hi, to this point works for my cluster, but then i have this problem:
kanibal@kubuntu:~$ mpiexec -n 2 cpi
problem with execution of cpi on nodo2: [Errno 2] No such file or directory
problem with execution of cpi on kabuntu: [Errno 2] No such file or directory

do you know a posible answer?
kanibal
kanibalv@gmail(NOSPAM).com
1:02 PM
Dr. Thiti said…
Kanibal,

Something might have gone wrong with your shared file system – perhaps the NFS shared directory was not properly mounted. I would suggest you log on to each machine manually and verify that “cpi” is accessible from each node.

--
Thiti.
5:49 PM
Kanibal said…
thank for write me back, but, since i don't have much experience with NFS, to this moment i can't make it to work properly, would you help me with the way for configuation of the nfs file system on the nodes.
i was thinking, if there is not much problem, a small tutorial for MPICH2 on ubuntu.
Thanks for your motivation.

kanibal.-
12:56 AM
Kanibal said…
finally work's...
hi, I made a small tutorial, please tell me how it is... (specially my english)
the link is:
http://kanibalv.blogspot.com/
"Installation and configuration of MPICH2 for a Beowulf Cluster".

thank's for all...

kanibal..
11:15 AM
Post a Comment

Popular posts from this blog

Padonc's "Bought from Mae Hong Son" T-shirts -- Interesting Souvenir from Mae Hong Son

Padonc's "Bought from Mae Hong Son" T-shirts -- Interesting Souvenir from Mae Hong Son The "I love New York" t-shirt and the series of souvenir was one of my impressive memory from my previous trips to New York City. I got this same kind of impression when I was browsing by the shops by the Jong Kam Walking Street in the city of Mae Hong Son in the second night of the trip. I didn't look at the seller when I caught the sight of these T-shirts. The seller greeted me and, surprisingly, she was P'Kwan, my senior alumni from Chulalongkorn University, who lives in Mae Hong Son and gave me a warm welcome when my family and I went for a dinner at Fern Restaurant in Mae Hong Son town the other night. Note: Pi or P' (its abbreviation) is a Thai word to call an elder sister or brother. Here are the pictures: P'Kwan and Her Shop The T-shirts P'Kwan and the Shop P'Kay, P'Kwan's Husband, at Another Shop Keywords: trips , mae-hong-son , thai
1 comment
Read more

Tip: Converting a CSV File into XML by Excel

I came across a problem to convert a historical data in comma-separated format (CSV) into a modern & more useful XML format. There are many ways to do such conversion. The most obvious one is to do it programmatically by writing a program to read CSV data and exporting it to an XML file by means of the Simple API for XML (SAX) or the Document Object Model (DOM) libraries. But I was too lazy to do it. I knew that Microsoft Excel 2003 supports exporting worksheets into XML data. I tried it out but found that it is not that simple -- we have to define an XML schema and bind each element to each column of the worksheet. So I went on researching. Finally, I found a very useful tool on MSDN: Excel 2003 XML Tools Add-in Version 1.1 ( download ). The Excel Add-in's "XML Tools -> Convert a Range to an XML List" menu command allows us to simply select a range in a worksheet (with or without first row as column names) -- with the advanced options to specify the XML root
Post a Comment
Read more

It’s been eight years and here comes COVID-19 in early 2020

This blog was last posted in 2008 before Facebook and LinkedIn went mainstream. I have since then socialized and posted a lot on Facebook and LinkedIn. During the global COVID-19 crisis in early 2020 people dramatically change the way they live and work since they day I last posted on this blog. They stay a lot at home during the great lockdown to prevent the spreading of the coronavirus. Here is a summary of my online presence: Facebook: http://fb.com/drthitiv LinkedIn: http://linkedin.com/in/thitiv LINE: http://line.thiti.it Bangkok, THAILAND April 22, 2020 
Post a Comment
Read more