[NLPL Task Force (A)] rolling your own BERT (and maybe ELMo) on Saga

[Stephan Oepen]

hi filip and antti,

for your forthcoming BERT tutorial, i was wondering about two things: (1)
what software environment do you require?  should we try pre-installing
something on Saga?  specifically, which framework do you assume for the
multi-gpu part?

and, more content-wise: (2) filip and i had previously talked about the
possibility of andrey kutuzov joining for part of your slot to share some
of his experience on training language-specific ELMo models on Saga, see

http://wiki.nlpl.eu/index.php/Vectors/elmo/tutorial

i checked with andrey last week, who would be prepared to use about twenty
minutes on ELMo, if you felt you could afford it time-wise in your
80-minute slot?

which reminds me, could i suggest that you also consider putting something
like your BERT recipe on the NLPL wiki?  i suspect filip has a wiki account
already, not sure about you, antti?  i could easily set you up for write
access to the wiki, and i wlike to collect materials there as much as
possible :-).

looking forward to seeing you in the mountains!

oe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nlpl.eu/archives/infrastructure/attachments/20200127/48f8f42c/attachment.htm>