[NLPL Task Force (A)] rolling your own BERT (and maybe ELMo) on Saga

[Filip Ginter]

Hi

Ah, the program changed, last time I heard we had 60min, now it is 80.
Surely we can try squeeze Andrei in, welcome on board!

Antti can tell about the exact GPU stuff needed. We will run the tutorial
on puhti since this is a tried and tested environment for us, and we have
little time to prepare, so we play it safe. But Antti can tell what it
takes to run the BERT code.

F



On Mon, Jan 27, 2020 at 3:07 PM Stephan Oepen <oe at ifi.uio.no> wrote:

> hi filip and antti,
>
> for your forthcoming BERT tutorial, i was wondering about two things: (1)
> what software environment do you require?  should we try pre-installing
> something on Saga?  specifically, which framework do you assume for the
> multi-gpu part?
>
> and, more content-wise: (2) filip and i had previously talked about the
> possibility of andrey kutuzov joining for part of your slot to share some
> of his experience on training language-specific ELMo models on Saga, see
>
> http://wiki.nlpl.eu/index.php/Vectors/elmo/tutorial
>
> i checked with andrey last week, who would be prepared to use about twenty
> minutes on ELMo, if you felt you could afford it time-wise in your
> 80-minute slot?
>
> which reminds me, could i suggest that you also consider putting something
> like your BERT recipe on the NLPL wiki?  i suspect filip has a wiki account
> already, not sure about you, antti?  i could easily set you up for write
> access to the wiki, and i wlike to collect materials there as much as
> possible :-).
>
> looking forward to seeing you in the mountains!
>
> oe
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nlpl.eu/archives/infrastructure/attachments/20200127/f7609ea7/attachment.htm>