[Beowulf] Storage Best Practices

[Richter, Brian J {BIS}]

Hey All,

I was hoping to get some recommendations for Storage. Last year we set up our first HPC and I'm looking for a good strategy moving forward for Storage. We set up a dedicated space on the cluster for Storage that has 5.5 TB of space. This space can be quickly chewed up depending on the project the business is working on. Do you guys typically do a retention policy on the cluster to ensure there is always enough space? I was thinking anything older than a month should be cleaned up.

Thanks!

Brian J. Richter
Global R&D Senior Analyst * Information Technology
617 W Main St, Barrington, IL 60010
Office: 847-304-2356 * Mobile: [phone]
***@pepsico.com<mailto:***@pepsico.com>

[Skylar Thompson]

For our larger groups, we'll meet with them regularly to discuss their
space usage (and other IT needs). Even that's unlikely to be frequent
enough, so we direct usage alerts to their designated "data manager" if
they're getting close to running out of space. Aside from regularly
clearing out scratch space, though, we try to stay out of deletion
decisions, to avoid problems stemming from miscommunication between the
group and IT, or even within the group (left hand doesn't know what the
right hand is doing).

Skylar

On Mon, Feb 19, 2018 at 6:12 AM, Richter, Brian J {BIS} <

Post by Richter, Brian J {BIS}
Hey All,
I was hoping to get some recommendations for Storage. Last year we set up
our first HPC and I’m looking for a good strategy moving forward for
Storage. We set up a dedicated space on the cluster for Storage that has
5.5 TB of space. This space can be quickly chewed up depending on the
project the business is working on. Do you guys typically do a retention
policy on the cluster to ensure there is always enough space? I was
thinking anything older than a month should be cleaned up.
Thanks!
*Brian J. Richter*
Global R&D Senior Analyst *•* Information Technology
617 W Main St, Barrington, IL 60010
Office: 847-304-2356 <(847)%20304-2356> *• *Mobile: [phone]
<(847)%20305-6306>
_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

[Tim Cutts]

No policy here. People can keep stuff as long as they like. I don’t agree with that lack of policy, but that’s where we are.

We did propose a 90 day limit, about 10 years ago. It lasted about, er, 90 days, before faculty started screaming. ☺

Tim

On 19/02/2018, 15:31, "Beowulf on behalf of Skylar Thompson" <beowulf-***@beowulf.org<mailto:beowulf-***@beowulf.org> on behalf of ***@gmail.com<mailto:***@gmail.com>> wrote:

For our larger groups, we'll meet with them regularly to discuss their space usage (and other IT needs). Even that's unlikely to be frequent enough, so we direct usage alerts to their designated "data manager" if they're getting close to running out of space. Aside from regularly clearing out scratch space, though, we try to stay out of deletion decisions, to avoid problems stemming from miscommunication between the group and IT, or even within the group (left hand doesn't know what the right hand is doing).
Skylar

On Mon, Feb 19, 2018 at 6:12 AM, Richter, Brian J {BIS} <***@pepsico.com<mailto:***@pepsico.com>> wrote:
Hey All,

I was hoping to get some recommendations for Storage. Last year we set up our first HPC and I’m looking for a good strategy moving forward for Storage. We set up a dedicated space on the cluster for Storage that has 5.5 TB of space. This space can be quickly chewed up depending on the project the business is working on. Do you guys typically do a retention policy on the cluster to ensure there is always enough space? I was thinking anything older than a month should be cleaned up.

Thanks!

Brian J. Richter
Global R&D Senior Analyst • Information Technology
617 W Main St, Barrington, IL 60010
Office: 847-304-2356<tel:(847)%20304-2356> • Mobile: [phone]<tel:(847)%20305-6306>
***@pepsico.com<mailto:***@pepsico.com>



_______________________________________________
Beowulf mailing list, ***@beowulf.org<mailto:***@beowulf.org> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

[Adam DeConinck]

The best success I've seen is a mix of strategies: some amount storage
that's considered "permanent" with aggressive quota limits (per user
or per project), plus a larger no-quota space that's cleaned up based
on some retention policy. I've used this basic scheme in small
environments, and also seen it used successfully in large institutions
(where I was not directly responsible for storage).

That said, there's no way to make everyone happy with this -- people
will complain when a file is deleted unexpectedly, or when they hit
their quota, or both. Time-based retention policies that work by
automatically checking file modification time also invite users to
write "touch" scripts to try to fool the cleaner. It can turn into a
fun game. But I've also used these kinds of policies to justify new
storage purchases, and make users into allies in funding them.

On a lighter note, this is a good excuse for me to link my favorite
James Cuff post ;-) -
http://blog.jcuff.net/2011/12/cautionary-tale-about-storage-and.html

Cheers,
Adam


On Mon, Feb 19, 2018 at 6:12 AM, Richter, Brian J {BIS}

Post by Richter, Brian J {BIS}
Hey All,
I was hoping to get some recommendations for Storage. Last year we set up
our first HPC and I’m looking for a good strategy moving forward for
Storage. We set up a dedicated space on the cluster for Storage that has 5.5
TB of space. This space can be quickly chewed up depending on the project
the business is working on. Do you guys typically do a retention policy on
the cluster to ensure there is always enough space? I was thinking anything
older than a month should be cleaned up.
Thanks!
Brian J. Richter
Global R&D Senior Analyst • Information Technology
617 W Main St, Barrington, IL 60010
Office: 847-304-2356 • Mobile: [phone]
_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

_______________________________________________
Beowulf mailing list, ***@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mail

[Adam DeConinck]

The best success I've seen is a mix of strategies: some amount storage
that's considered "permanent" with aggressive quota limits (per user
or per project), plus a larger no-quota space that's cleaned up based
on some retention policy. I've used this basic scheme in small
environments, and also seen it used successfully in large institutions
(where I was not directly responsible for storage).

That said, there's no way to make everyone happy with this -- people
will complain when a file is deleted unexpectedly, or when they hit
their quota, or both. Time-based retention policies that work by
automatically checking file modification time also invite users to
write "touch" scripts to try to fool the cleaner. It can turn into a
fun game. But I've also used these kinds of policies to justify new
storage purchases, and make users into allies in funding them.

On a lighter note, this is a good excuse for me to link my favorite
James Cuff post ;-) -
http://blog.jcuff.net/2011/12/cautionary-tale-about-storage-and.html

On Mon, Feb 19, 2018 at 6:12 AM, Richter, Brian J {BIS}

Post by Richter, Brian J {BIS}
Hey All,
I was hoping to get some recommendations for Storage. Last year we set up
our first HPC and I’m looking for a good strategy moving forward for
Storage. We set up a dedicated space on the cluster for Storage that has 5.5
TB of space. This space can be quickly chewed up depending on the project
the business is working on. Do you guys typically do a retention policy on
the cluster to ensure there is always enough space? I was thinking anything
older than a month should be cleaned up.
Thanks!
Brian J. Richter
Global R&D Senior Analyst • Information Technology
617 W Main St, Barrington, IL 60010
Office: 847-304-2356 • Mobile: [phone]
_______________________________________________
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf

_______________________________________________
Beowulf mailing list, ***@beowulf.org sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/be

[Richter, Brian J {BIS}]

I'd like to thank everyone for your responses!

Brian J. Richter
Global R&D Senior Analyst * Information Technology
617 W Main St, Barrington, IL 60010
Office: 847-304-2356 * Mobile: [phone]
***@pepsico.com<mailto:***@pepsico.com>


From: Richter, Brian J {BIS}
Sent: Monday, February 19, 2018 8:12 AM
To: '***@beowulf.org' <***@beowulf.org>
Subject: Storage Best Practices

Hey All,

I was hoping to get some recommendations for Storage. Last year we set up our first HPC and I'm looking for a good strategy moving forward for Storage. We set up a dedicated space on the cluster for Storage that has 5.5 TB of space. This space can be quickly chewed up depending on the project the business is working on. Do you guys typically do a retention policy on the cluster to ensure there is always enough space? I was thinking anything older than a month should be cleaned up.

Thanks!

Brian J. Richter
Global R&D Senior Analyst * Information Technology
617 W Main St, Barrington, IL 60010
Office: 847-304-2356 * Mobile: [phone]
***@pepsico.com<mailto:***@pepsico.com>