[NLPL Task Force (A)] Abel for teaching

[Stephan Oepen]

hi again, sara,

we have received confirmation in the meantime that Abel is expected to
remain operational (on a 'best-effort' basis) until at least
mid-november, and that most users are expected to migrate to Saga in
september.  i will then formally ask USIT to acknowledge that we are
planning on this teaching use case under the NLPL umbrella, and
hopefully they will confirm that this is a good idea.

most of the NMT infrastructure in NLPL is installed on Taito only,
however, reflecting a division of labor we had agreed on a couple of
years ago.  there is ONMT (0.2.1) available on Abel, though, and i too
will have a student this fall experimenting with that.  so, if that
sound like sufficient pre-installed software support to you, maybe all
is good already?  if not, we should discuss what software you would
ideally like to use.

best wishes, oe


On Wed, Aug 14, 2019 at 1:27 PM Sara Stymne <sara.stymne at lingfil.uu.se> wrote:
>
> Hi Stephan,
>
> Thanks for your fast answer!
>
> I forgot to mention that the Swedish option unfortunately didn't work out. We were not allowed to use it for teaching.
>
> We need one NMT toolkit. The labs and projects will actually be run by Gongbo Tang, one of our PhD students, who is in China this week, so I don't know his preference for toolkit. I'll get back with that info as soon as I know, which is most likely next week. Hopefully one of the toolkits already installed through NLPL, will work.
>
> As for system, I don't have a clear preference. Maybe Abel is a bit more safe, since it should not be affected by possible delays, I guess. And it would be convenient that a lot of toolkits and corpora are already available there.
>
> Best,
> Sara
>
>
>
> 14 aug 2019 kl. 11:58 skrev Stephan Oepen <oe at ifi.uio.no>:
>
> > hi again, sara,
> >
> > i am copying the NLPL infrastructure task force and joakim (member of
> > the steering group), as i find this an interesting question, where
> > ideally we should come up with a good solution for future occasions!
> >
> > you had mentioned before the summer, that you were also looking for
> > gpu acccess from one of the swedish centers, but it sounds as if you
> > are currently not very optimistic about that?
> >
> > the situation in norway is as follows: the Abel successor (Saga) is
> > completing its successful trial period these days and will likely go
> > in production in early september.  Saga has 32 modern gpus, so with a
> > bit of luck there should be some capacity between mid-september and
> > early november (i expect we will perceive a shortage of gpu capacity
> > again soon enough, though, possibly well before the end of the year).
> >
> > at the same time, Abel will probably remain in production until
> > sometime in november, and one could also speculate on the transition
> > of users to Saga freeing up gpu capacity there.  my impression from
> > talking to yves earlier this year is that the old K20 gpus actually
> > are fully sufficient for decent job sizes and training times using
> > OpenNMT.
> >
> > for either of these two scenarios, i am thinking we should request a
> > kind of confirmation from the NLPL infrastructure task force and from
> > the system administrators (primarily USIT, both for Abel and Saga),
> > for example regarding general availability of Saga by mid-september or
> > remaining life time for Abel until november.
> >
> > among the two candidate scenarios above, would you have a clear
> > preference?  any comments or words of wisdom from other infrastructure
> > members?
> >
> > in terms of software, what would you need?
> >
> > best wishes, oe
> >
> > On Wed, Aug 14, 2019 at 11:41 AM Sara Stymne <sara.stymne at lingfil.uu.se> wrote:
> >>
> >> Hi Stephan, and others!
> >>
> >> I just wanted to check if there is any progress on this.
> >>
> >> Best,
> >> Sara
> >>
> >>
> >>
> >> 20 jun 2019 kl. 12:56 skrev Sara Stymne <sara.stymne at lingfil.uu.se>:
> >>
> >> The course will run September to the beginning of November. I expect them to need gpu from around mid September. There will be around 20 students, but they will be working in pairs, so around 10 student groups.
> >>
> >> Best,
> >> Sara
> >>
> >> The course will run September to the beginning of November. I expect them to need gpu from around mid September. There will be around 20 students, but they will be working in pairs, so around 10 student groups.
> >>
> >> Best,
> >> Sara
> >>
> >> ---- Stephan Oepen skrev ----
> >>
> >> hi sara,
> >>
> >> i am taking the liberty of copying the NLPL steering group (which
> >> subsumes the infrastructure task force), because you raise an
> >> interesting question: availability of gpu resources, and their use in
> >> a teaching scenario.
> >>
> >>> I am teaching our MT course this autumn, and we're planning to introduce more NMT. To have any type of reasonable lab and project assignments, we will need GPU's, though. I am trying to figure out how to manage that. I have also checked with a Swedish cluster. But if that fails, do you think it would it be OK to use Abel within NLPL for our master students, and have them do some NMT tasks there?
> >>
> >> in principle, i would say: yes.  but just now it is hard to predict
> >> how things will play out in practice.  Abel has few and old gpu nodes,
> >> and this past semester there have been periods when one had to wait
> >> for at least several days before getting a gpu job to run.  Abel will
> >> be replaced by a new system, called Saga, starting (hopefully) already
> >> in august this year.  this may mean that Abel load will be
> >> substantially lighter in the fall semester, but it may also mean that
> >> Abel becomes unavailable before the end of the year.
> >>
> >> so, one could then ask the same question about Saga.  in principle, i
> >> would again say that NLPL is intended to facility HPC use in research
> >> and (advanced) teaching.  the NMT software and data you will want to
> >> use are likely part of the NLPL infrastructure (and if not, maybe
> >> should be); teachning NMT, as you point out, is hardly feasible
> >> without gpu access.  so what are we going to do, if not slip these
> >> students in on the NLPL systems?
> >>
> >> exactly over which time period will your class need gpu resources?
> >> and do you have an estimate of the number of active students?
> >>
> >> best wishes, oe
> >> hi sara,
> >>
> >> i am taking the liberty of copying the NLPL steering group (which
> >> subsumes the infrastructure task force), because you raise an
> >> interesting question: availability of gpu resources, and their use in
> >> a teaching scenario.
> >>
> >> I am teaching our MT course this autumn, and we're planning to introduce more NMT. To have any type of reasonable lab and project assignments, we will need GPU's, though. I am trying to figure out how to manage that. I have also checked with a Swedish cluster. But if that fails, do you think it would it be OK to use Abel within NLPL for our master students, and have them do some NMT tasks there?
> >>
> >>
> >> in principle, i would say: yes.  but just now it is hard to predict
> >> how things will play out in practice.  Abel has few and old gpu nodes,
> >> and this past semester there have been periods when one had to wait
> >> for at least several days before getting a gpu job to run.  Abel will
> >> be replaced by a new system, called Saga, starting (hopefully) already
> >> in august this year.  this may mean that Abel load will be
> >> substantially lighter in the fall semester, but it may also mean that
> >> Abel becomes unavailable before the end of the year.
> >>
> >> so, one could then ask the same question about Saga.  in principle, i
> >> would again say that NLPL is intended to facility HPC use in research
> >> and (advanced) teaching.  the NMT software and data you will want to
> >> use are likely part of the NLPL infrastructure (and if not, maybe
> >> should be); teachning NMT, as you point out, is hardly feasible
> >> without gpu access.  so what are we going to do, if not slip these
> >> students in on the NLPL systems?
> >>
> >> exactly over which time period will your class need gpu resources?
> >> and do you have an estimate of the number of active students?
> >>
> >> best wishes, oe
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> När du har kontakt med oss på Uppsala universitet med e-post så innebär det att vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/
> >>
> >> E-mailing Uppsala University means that we will process your personal data. For more information on how this is performed, please read here: http://www.uu.se/en/about-uu/data-protection-policy
>