Only the parallelized programs should be compiled with the parallel Machine files. E.g. on SP2, use Machine.rs6000 to compile everything but sander and gibbs. When making source for DOS, I got the following error:
% make bigsource ... Making source: machine is DOS w/ NDP-Fortran-486 CPP: Undefined variable.
You should be using mksrc to create source for a machine other than the one you are running on, and at the same time, your MACHINE file should be appropriate for the machine you are on (which solves the CPP problem).
I too would question whether -DPVM will work straight out, although in the past I have been able to get it to work (for example on a network of SGI indigos). However, in general I tend to run directly under MPI since this is more efficient. We have access to a 2-processor DEC AXP SMP and I have been able to compile and run the MPICH MPI implementation on this machine. For more information about obtaining this release see
http://www.mcs.anl.gov/home/lusk/mpich/index.html A