[Beowulf] HPC Workflows

[m***@chem.leidenuniv.nl]

Yeah, I often thing some people are using the letters HPC as in 'high
profile computing' nowadays. The diluting effect I mentoined a few posts
ago.

Actually LOT of HPC admin folks I know are scientists, scientificly active
and tightly coupled to scientists in groups and they were doing DevOps
even before it got that fancy name.

Here at my uni I find it is *regular* IT that needed and is (indeed in a
strange herdy way) adjusting to the pace and workings of scientists and
science. They are using old concepts that have recently relabeled into
configuration management or stack deployments etc. (as if we used to
install them hundreds of HPC nodes all by hand or something?) Things there
get re introduced by them commercial-here-is-my-bill-consultants with a
fancy gui klickemy-thingy oh and an icon. Let me tell you, now they are
doing them stand up comedy things there with their scum mates every day
and are talking about 'fragile' software and saying things like 'moving
fast breaking things' ?!

Ha, all i can say is i am keeping my distance there cause there is a lot
of branding and marketing buzzword speak involved and it is giving me a
rash ! Back to running my Molecular Quantum Dynamics I am.

m.
--
mark somers
tel: [phone]
mail: ***@chem.leidenuniv.nl
web: http://theorchem.leidenuniv.nl/people/somers


_______________________________________________
Beowulf mailing list, ***@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.

[John Hanks]

For me personally I just assume it's my lack of vision that is the problem.
I was submitting VMs as jobs using SGE well over 10 years ago. Job scripts
that build the software stack if it's not found? 15 or more. Never occurred
to me to call it "cloud" or "containerized", it was just a few stupid
scripts to solve some specific problem we had. I look at containers and
cloud now and just don't get it. Early in my career I had a mentor who was
from the IBM mainframe world. I recall excitedly explaining what I was
playing around with with early Xen versions and he said "Yeah, we've been
doing that for a long time." Now it's my turn to say "Yeah, I was doing
that years ago" and scratch my head at what all the fuss is about. Such are
the effects of the ravages of time.

Keeping distance from IT is always a good idea. The first rule of
interacting with IT is: "IT is not here to solve your problem, IT is here
to justify more IT. If your problem is solved, then it is the result of
random chance, do not look for patterns." Best to sit as close to the
scientists as possible.

griznog

Post by m***@chem.leidenuniv.nl
Yeah, I often thing some people are using the letters HPC as in 'high
profile computing' nowadays. The diluting effect I mentoined a few posts
ago.
Actually LOT of HPC admin folks I know are scientists, scientificly active
and tightly coupled to scientists in groups and they were doing DevOps
even before it got that fancy name.
Here at my uni I find it is *regular* IT that needed and is (indeed in a
strange herdy way) adjusting to the pace and workings of scientists and
science. They are using old concepts that have recently relabeled into
configuration management or stack deployments etc. (as if we used to
install them hundreds of HPC nodes all by hand or something?) Things there
get re introduced by them commercial-here-is-my-bill-consultants with a
fancy gui klickemy-thingy oh and an icon. Let me tell you, now they are
doing them stand up comedy things there with their scum mates every day
and are talking about 'fragile' software and saying things like 'moving
fast breaking things' ?!
Ha, all i can say is i am keeping my distance there cause there is a lot
of branding and marketing buzzword speak involved and it is giving me a
rash ! Back to running my Molecular Quantum Dynamics I am.
m.
--
mark somers
tel: [phone]
web: http://theorchem.leidenuniv.nl/people/somers
_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

[Tim Cutts]

Ho ho. Yes, there is rarely anything completely new. Old ideas get dusted off, polished up, and packaged slightly differently. At the end of the day, a Dockerfile is just a script to build your environment, but it has the advantage now of doing it in a reasonably standard way, rather than whatever random method any of us might have come up with independently in the past. Adding central repositories and allowing you to base one Dockerfile on top of another are nice additions. Neither of those ideas is new either, of course.

I agree with your last sentence, we in scientific IT definitely need to stick very close to the scientists. But I don’t necessarily agree with the whole of the last paragraph. IT may appear to be there to justify more IT, but I think that’s a vicious circle. Many organisations see IT (whether scientific or enterprise) as a cost centre rather than as a strategic tool to meet their goals. They become forced into justifying their own existence, and of course the political game then becomes they have to seek to expand in order to stay the same size, otherwise they will be cut. Many years ago I listened to a talk by Joe Baguley from VMware, who said he could determine which was the case or not when talking to a customer with a single question, which was: “Who does your CIO report to?” CEO: good, CFO: bad

I suspect there’s some truth in that.

Regards,

Tim

For me personally I just assume it's my lack of vision that is the problem. I was submitting VMs as jobs using SGE well over 10 years ago. Job scripts that build the software stack if it's not found? 15 or more. Never occurred to me to call it "cloud" or "containerized", it was just a few stupid scripts to solve some specific problem we had. I look at containers and cloud now and just don't get it. Early in my career I had a mentor who was from the IBM mainframe world. I recall excitedly explaining what I was playing around with with early Xen versions and he said "Yeah, we've been doing that for a long time." Now it's my turn to say "Yeah, I was doing that years ago" and scratch my head at what all the fuss is about. Such are the effects of the ravages of time.
Keeping distance from IT is always a good idea. The first rule of interacting with IT is: "IT is not here to solve your problem, IT is here to justify more IT. If your problem is solved, then it is the result of random chance, do not look for patterns." Best to sit as close to the scientists as possible.