[NLPL Task Force (A)] [rt.uio.no #3406598] usage statistics for NLPL modules

[Thomas Röblitz via RT]

Might also be interesting to check how often these unique users use modules and if they return or just use it for a certain period ... lots of analytics possible ;), and maybe which are popular (the latter could be interesting for us to know better what general software is essential for users).

Thomas

On 2019-05-13 22:59:49, oe wrote:
> great, thanks a lot, bjoern-helge!  that file is actually slightly
> longer than i would have anticipated, but then i suppose log files
> tend to grow quickly :-).
> 
> the strings sent via logger(1) actually provide a fully-qualified date
> value, so sorting by years seems relatively straightforward:
> 
> [oe at login-0-0 ~]$ grep 'DATE=2017-' nlpl-modules.log |wc -l
> 78
> [oe at login-0-0 ~]$ grep 'DATE=2018-' nlpl-modules.log |wc -l
> 18019
> [oe at login-0-0 ~]$ grep 'DATE=2019-' nlpl-modules.log |wc -l
> 97373
> 
> likewise, the trend for unique users also points in a healthy
> direction:
> 
> [oe at login-0-0 ~]$ grep 'DATE=2017-' nlpl-modules.log | awk -F, '{print
> $2}' | sort -u | wc -l
> 3
> [oe at login-0-0 ~]$ grep 'DATE=2018-' nlpl-modules.log | awk -F, '{print
> $2}' | sort -u | wc -l
> 29
> [oe at login-0-0 ~]$ grep 'DATE=2019-' nlpl-modules.log | awk -F, '{print
> $2}' | sort -u | wc -l
> 42
> 
> to help put things into perspective, could you easily count for us the
> total number of module actions on Abel in, say, DATE=2019-03 and
> DATE=2019-034?
> 
> warmest thanks again!  oe
> 
> On Mon, May 13, 2019 at 9:58 AM Bjørn-Helge Mevik via RT
> <hpc-drift at usit.uio.no> wrote:
> >
> > On 2019-05-10 14:05:16, oe wrote:
> >
> > > any chance someone could make accessible to me the relevant log
> > > files
> > > (preferably going back to, say, august 2018)?  or alternatively
> > > grep(1) out
> > > all occurrences of the prefix ‘nlpl-’ in module names and point me
> > > to the
> > > result?
> >
> > Sure, I've grepped for "=nlpl-" in our log files now.  I put the
> > result in ~oe/nlpl-modules.log.  This covers all usage on login- or
> > compute-nodes (but not appnodes).
> >
> > Note that syslog doesn't include the year in the date format, so you
> > might have to do a little checking to find out which year a given
> > date belongs to.  The logs are in date order, so it shouldn't be too
> > difficult, though.
> >
> > --
> > Regards,
> > Bjørn-Helge Mevik, dr. scient,
> > Department for Research Computing, USIT
> >