From 610b5ef4928b81ab4eecbcdf2ac38197614f0953 Mon Sep 17 00:00:00 2001 From: Thomas Hochstein Date: Fri, 17 Sep 2010 21:16:51 +0200 Subject: [PATCH] Add documentation. Add doc/ Add doc/README Add doc/INSTALL Signed-off-by: Thomas Hochstein --- doc/INSTALL | 128 ++++++++++++++++++++++++++++++++++++++++++++++++++++ doc/README | 94 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 222 insertions(+) create mode 100644 doc/INSTALL create mode 100644 doc/README diff --git a/doc/INSTALL b/doc/INSTALL new file mode 100644 index 0000000..225fdbf --- /dev/null +++ b/doc/INSTALL @@ -0,0 +1,128 @@ +NewsStats 0.1 (c) 2010 Thomas Hochstein + +NewsStats is a software package for gathering statistical data live +from a Usenet feed and subsequent examination. + +This script package is free software; you can redistribute it and/or +modify it under the terms of the GNU Public License as published by +the Free Software Foundation. + +--------------------------------------------------------------------- + +INSTALLATION INSTRUCTIONS + +1) Install the scripts + + * Download the current version of NewsStats from + . + + * Untar it into a directory of your choice: + + # tar -xzf newsstats-nn.tar.gz + + Scripts in this path should be executable by the news user. + +2) Configuration + + * Copy the sample configuration file newsstats.conf.sample to + newsstats.conf and modify it for your purposes: + + # cp newsstats.conf.sample newsstats.conf + # vim newsstats.conf + + a) Mandatory configuration options + + * DBDriver = mysql + Database driver used; currently only mysql is supported. + + * DBHost = localhost + The host your mysql server is running on. + + * DBUser = + The username to connect to the database server. + + * DBPw = + Matching password for your username. + + * DBDatabase = newsstats + Database name. + + NewsStats will use those credentials to connect to your mysql + installation. + + * DBTableRaw = raw_de + Table holding raw statistical data. + + * DBTableGrps = groups_de + Table holding data on postings per group. + + b) Optional configuration options + + * TLH = de + Limit examination to that top-level hierarchy. + +3) Database (mysql) setup + + * Setup your database server with a username, password and + database matching the NewsStats configuration (see 2 a). + + * Start the installation script: + + # install/install.pl + + It will setup the necessary database tables and display some + information on the next steps. + +4) Feed (INN) setup + + You have to setup an INN feed to feedlog.pl. + + * Edit your 'newsfeeds' file and insert something like + + ## gather statistics for NewsStats + newsstats! + :!*,de.* + :Tc,WmtfbsPNH,Ac:/path/to/feedlog.pl + + * You should only feed that hierarchy (those hierarchies ...) to + feedlog.pl you'll want to cover with your statistical + examination. It may be a good idea to setup different feeds (to + different databases ...) for different hierarchies. + + * Please double check that your path to feedlog.pl is correct and + feedlog.pl can be executed by the news user + + * Check your 'newsfeeds' syntax: + + # ctlinnd checkfile + + * Reload 'newsfeeds': + + # ctlinnd reload newsfeeds 'Adding newsstats! feed' + + * Watch your 'news.notice' and 'errlog' files: + + # tail -f /var/log/news/news.notice + ... + # tail -f /var/log/news/errlog + + Everything should be going smoothly now. + + * If INN is spewing error messages to 'errlog' or reporting + continous respaws of feedlog.pl to 'news.notice', stop your feed: + + # ctlinnd drop 'newsstats!' + + and investigate. 'errlog' may be helpful here. + + * You can restart the feed with + + # ctlinnd begin 'newsstats!' + + later. + +You should be done now. + +Just have a look at your raw data (DBTableRaw). It should now start to +fill up. + diff --git a/doc/README b/doc/README new file mode 100644 index 0000000..9bea871 --- /dev/null +++ b/doc/README @@ -0,0 +1,94 @@ +NewsStats 0.1 (c) 2010 Thomas Hochstein + +NewsStats is a software package for gathering statistical data live +from a Usenet feed and subsequent examination. + +This script package is free software; you can redistribute it and/or +modify it under the terms of the GNU Public License as published by +the Free Software Foundation. + +--------------------------------------------------------------------- + +What's that? + + There's a multitude of tools for the statistical examination of + newsgroups: number of postings month or per person, longest + threads, and so on (see + [German language] for an incomplete list). Most of them use a per- + newsgroup approach while NewsStats is hierarchy oriented. + + NewsStats will accumulate data from a live INN feed, allowing you + to process the saved information later on. + +Workflow + + NewsStats saves overview data and complete headers of (all) + incoming postings to a (MySQL) database in real time. + + That raw data will be regularly - e.g. monthly - processed to a + second set of database tables each dedicated to a certain + statistical aspect, e.g. number of postings per group per month. + + Several kinds of reports can then be generated from those result + tables. + +Prerequisites + + NewsStats is written in Perl (5.8.x and above) and makes use of a + MySQL database, so you'll need Perl, some modules, mysql and, of + course, an INN. + + * Perl 5.8.x with standard modules + - Cwd + - File::Basename + - Sys::Syslog + + * Perl modules form CPAN + - Cofing::Auto + - Date::Format + - DBI + + * mysql 5.0.x + + * working installation of INN + +Installation instructions + + See INSTALL. + +Getting Started + + 'feedlog.pl' will continuously feed raw data to your raw data + table. See the feedlog.pl man page for more information. + + You can process that data via 'gatherstats.pl'; currently only the + tabulation of postings per group per month is supported. More to + come. See the gatherstats.pl man page for more information. + + Report generation is handled by specialised scripts for each + report type. Currently only reports on the number of postings per + group per month are supported; you can use 'groupstats.pl' for + that. See the groupstats.pl man page for more information. + +Reporting Bugs + + You can report bugs or feature requests to the author by using the + bug tracker at . + +More Information + + This program is maintained using the Git version control system. + You may clone to check + out the current development tree or browse it on the web via + . + +Related projects + + is a site were data gathered via NewsStats + is used for a graphical presentation of activity in the de.* + Usenet hierarchy over the years. + +Author + + Thomas Hochstein + -- 2.20.1