NimNews

Immature Newsgroup NNTP server using Nim and SQLite

The goal of this server is to provide a flexible NNTP interface to whatever you want. Articles are stored in SQLite and the server itself is simple enough to be flexible if you need it to be. It is not designed to peer with Usenet although it could be considered for a future improvement. No policy is implemented and all commands are accepted even to logged-out users

Implementation status and RFCs

• RFC-977 should be implemented in full except distributions in NEWNEWS and NEWGROUPS
• RFC-2980 to be implemented (defining XOVER)
• RFC-3977 in implemented in part (missing XOVER kind of requests and many others)
• RFC-4642: STARTTLS extension is drafted (not tested, might not be secure)
• RFC-4643: AUTH extension is drafted with USER/PASS, SASL PLAIN (RFC-4616) and SASL SCRAM RFC-5802 (missing user database yet!)
• RFC-850: Message structure, control messages, not implemented at all except very basic parsing of headers and body following mostly RFC-822 RFC-2822 and RFC-5322
• RFC-2919 and RFC-2369: List-Id and other List headers on feed email list
• RFC-2033 (LMTP), RFC-821, RFC-1869 (SMTP and ESMTP)
• RFC-2047 encoded words

Goals

• Newsgroup server which can serve as a backend to Web UI as well as public server for classic clients

• Open federation with other servers using SMTP as transfer protocol between instances using e-mail subscriptions

• Handles authentication so only verified accounts and verified e-mail address can POST messages, ensuring that there is no abuse

Architecture

NimNews works with a SMTP server to handle:

• outgoing e-mail feed subscriptions
• outgoing e-mail password notification (to mail password to users)
• incoming e-mail feed (to post to group alt.xyz from e-mail [email protected])

Connection is performed using standard LMTP.

     ,---(feed)---------> [Remote nimnews]
     |
     |
     v
[SMTP server] <-----> [SMTP] <--------> [Mail User Agent]
  ^      |
  |      |
  |      v
,--+---[LMTP]---.
|               |
|    nimnews    | <-------------------> [NNTP User Agent]
|               |
`---------------'
       ^
       |
       v
    [SQLite]

TODO

• Do not mangle Form header if DMARC is not enabled on the author domain

• Handle user permission, only allow posting if the From header matches the user name

• Handle authentication when feeding messages (the sending server should tell the receiving one that the newsgroup came from itself and not some random party, could be via specific DKIM)

• Handle NimNews issuing FEED EMAIL commands to servers it wants to receive feeds from (currently, the newsmaster has to do this manually)

• Add console command to CLI where NNTP prompt is provided in admin mode. useful to manage subscriptions and accounts.

• Include the SMTP server, probably as a process spawned from NimNews.

• Be binary safe, end line markers are probably to blame here.

More ideas

• HTTP interface to handle the registering and log-in process by e-mail. It might just be NimNews getting the NewsWeb domain name and integrating it with e-mails that are sent for login. Do not reuse the password sent by e-mail and invalidate them.

• Handle private groups

  • Initial message with specific control header creates the group
  • To belong to the group, a message must have a specific header with the initial group message-id. LMTP can receive mail for a group and generate this header.
  • Private groups are only accessible to their members
  • Groups are not encrypted (for now)
  • Control messages can add new members or remove members

• Secure messages

  • Each user is associated with keypairs
  • Private groups have a keypair
  • Members of the private groups have the group private key
  • Messages are encrypted with the group key, except some headers (the group idenifying header, the Path header, the message-id, group public key)
  • Group messages are signed by the sending member key
  • Servers do not have access to group membership and clients must advertise to the server the public group keys they wish to have access to
  • Banning a user works by changing the private key of a group but not notifying the banned user of the new key. The new key is encrypted for all non-banned users.

• Summarizing NNTP server

  • A separate server that gets feed from a classic NimNews server
  • Can have the private keys to some groups it needs to access
  • Takes all messages in groups and present them better
    • Provides summary of current members in private groups by collecting all control messages
    • Adds headers to existing messages to mark reactions (votes, emotes)
  • Can be configured to follow a specific moderator, and removes moderated content

Build

nimble install -d
nimble c src/nimnews

This will compile with -d:ssl, compiling without it might work but is not well tested.

Run

Try it out:

./gen-cert.sh
./run-exim.sh
./run-nimnews.sh

Options are:

Nimnews is a simple newsgroup NNTP server

Usage: nimnews [options]

Options:
  -h, --help            Print help
  -p, --port <port>     Specify a different port [default: 119]
  -d, --db <file>       Database file [default: ./nimnews.sqlite]
  -f, --fqdn <fqdn>     Fully qualified domain name
  -s, --secure          Indicates that the connection is already encrypted
  --admin               Indicates that every anonymous user is admin
  --log                 Log traffic
  --smtp <server>       Address of SMTP server to send e-mails
  --smtp-port <port>    Port to connect to the SMTP server [default: 25]
  --smtp-login <login>  Login for SMTP server
  --smtp-pass <pass>    Password for SMTP server
  --smtp-sender <email> Email address to send e-mails as
  --smtp-debug          Debug SMTP
  --lmtp-port <port>    Specify port for LMTP [default: 24]
  --lmtp-addr <addr>    Specify listen address for LMTP [default: 127.0.0.1]
  --lmtp-socket <file>  Socket file for LMTP
  --tls-port <port>     Port number for NNTPS [default: 563]
  --cert <pemfile>      PEM certificate for STARTTLS
  --skey <pemfile>      PEM secret key for STARTTLS

NewsWeb web interface

This package also contains the newsweb web interface that is a forum interface for the Web designed to work with nimnews.

Build

nim c -d:ssl src/web/newsweb

Run

./run-newsweb.sh

X-NIMNEWS Extension

Nimnews advertise X-NIMNEWS extension with the following commands:

FEED EMAIL

Syntax: FEED EMAIL [LIST] <[email protected]> [WILDMAT [<site-id>]]

The feed command registers a new feed using e-mail. A feed is a link with another system happening over e-mail using the provided e-mail address. If the LIST keyword is present, then mailing-list style distribution is assumed with list headers. mailing-list distribution might mangle the From header to pass DMARC.

A site-id may be specified, instructing not to feed articles that contains this id in the Path header.

An optional WILDMAT parameter can be specified. The wildmat pattern would match groups that need feeding. if not specified, only the current group will be fed.

Feeds may be cancelled by the system any time, but should not do so without notification, unless there is delivery errors.

To be accepted, the user must be logged-in and the provided e-mail address must be accepted for the given user. If the user is administrator, all addresses are accepted. If the user e-mail matches RFC-2142 postmaster@*, usenet@*, news@*, every e-mail within that domain is allowed in the FEED EMAIL command. Else, only the user e-mail is allowed.

Responses:

• 290 <num> Feed registered
• 412 No selected group if wildmat is not defined and no group is selected
• 480 Disallowed (user not matching address, not registered or not admin)

LIST FEEDS

Return a list of feeds registered by the current user. The list is formatted as follows:

<num> EMAIL <[email protected]> <wildmat-or-group> <site-id>

Responses:

• 295 List follows
• 480 User not registered

STOP FEED

Syntax: FEED STOP <num>

Stop a given feed (removing it)

Responses:

• 290 Feed stopped
• 490 No such feed
• 480 User not registered

LIST USERS

Return a list of users registered on the system. This is intended for interactive use and the list format is not defined.

Responses:

• 295 List follows
• 480 Disallowed unless the user is administrator

AUTHINFO X-LOGIN

Syntax: AUTHINFO X-LOGIN

Tells the server that this is a login attempt with valid credentials. Credentials should not be reset upon authentication failure. This should be followed by another AUTHINFO command.

Responses:

• 381 Continue with login attempt

AUTHINFO X-REGISTER

Syntax: AUTHINFO X-REGISTER <email>

Tells the server that this is a register/recovery attempt for the given e-mail address. Credentials should be sent to this address for future login.

Responses:

• 381 Registration in progress