Yet another P2P Messenger

Reading reviews and documentation about the language is not enough to learn how to write less useful applications.

It is imperative that you need to create something interesting to consolidate, so that you can use it in other tasks.

The article is aimed at beginners interested in the go language and peer-to-peer networks.
And for professionals who can offer intelligent ideas or constructively criticize.

I program for a long time with a different degree of immersion in java, php, js, python.
And each programming language is good in its field.

The main area for Go is the creation of distributed services, microservices.
Most often, microservice is a small program that performs its highly specialized functionality.

But microservices should still be able to communicate with each other, so the tool for creating microservices should allow you to easily and easily organize network interaction.
To check this, we will write an application that organizes a decentralized network of equal participants (Peer-To-Peer), the simplest is the p2p messenger (by the way, is there a Russian synonym for this word?).

In the code I actively invent bicycles and step on a rake in order to feel the golang, to receive constructive criticism and rational suggestions.

What we do

Feast (peer) - a unique copy of the messenger.

Our messenger should be able to:

• Find nearby peers
• Connect with other peers
• Encrypt data exchange with peers
• Receive messages from user
• Show messages to user

To make the puzzle a little more interesting, let's make it all go through one network port.

If you pull this port via HTTP, you will get a Reakt application that pulls the same port by establishing a web socket connection.

If you pull the port over HTTP from the local machine, then we show the banner.

If another peer connects to this port, then a permanent connection is established with end-to-end encryption.

Determine the type of incoming connection

To begin, open the port to listen and wait for new connections.

net.ListenTCP("tcp", tcpAddr) 

On the new connection we read the first 4 bytes.

We take a list of HTTP verbs and compare our 4 bytes with it.

Now we determine whether the connection is made from the local machine, and if not, then we answer with a banner and "hang up".

  buf, err := readWriter.Peek(4) /* обработка ошибки */ if ItIsHttp(buf) { handleHttp(readWriter, conn, p) } else { peer := proto.NewPeer(conn) p.HandleProto(readWriter, peer) } /* ... */ if !strings.EqualFold(s, "127") && !strings.EqualFold(s, "[::") { response.Body = ioutil.NopCloser(strings.NewReader("Peer To Peer Messenger. see https://github.com/easmith/p2p-messenger")) } 

If the connection is local, then we respond with a file corresponding to the request.

Here I decided to write the processing myself, although I could use the handler available in the standard library.

  // свой способ func processRequest(request *http.Request, response *http.Response) {/* много строчек кода */} // либо из страндартной библиотеки fileServer := http.FileServer(http.Dir("./front/build/")) fileServer.ServeHTTP(NewMyWriter(conn), request) 

If the path /ws requested, then we try to establish a websocket connection.

Since I’ve put together a bike in handling file requests, I’ll do the ws connection processing using the gorilla / websocket library .

To do this, create MyWriter and implement methods in it to match the interfaces http.ResponseWriter and http.Hijacker .

  // w - MyWriter func handleWs(w http.ResponseWriter, r *http.Request, p *proto.Proto) { c, err := upgrader.Upgrade(w, r, w.Header()) /* теперь работаем с соединением почти как с обычным сокетом */ } 

Peer Detection

To search for peers in the local network, we use the multicast UDP.

We will send packets with information about ourselves to Multicast IP address.

  func startMeow(address string, p *proto.Proto) { conn, err := net.DialUDP("udp", nil, addr) /* ... */ for { _, err := conn.Write([]byte(fmt.Sprintf("meow:%v:%v", hex.EncodeToString(p.PubKey), p.Port))) /* ... */ time.Sleep(1 * time.Second) } } 

And separately listen to all UDP packets from Multicast IP.

  func listenMeow(address string, p *proto.Proto, handler func(p *proto.Proto, peerAddress string)) { /* ... */ conn, err := net.ListenMulticastUDP("udp", nil, addr) /* ... */ _, src, err := conn.ReadFromUDP(buffer) /* ... */ // connectToPeer handler(p, peerAddress) } 

Thus, we declare ourselves and find out about the appearance of other peers.

One could organize this at the IP level and even in the official documentation of the IPv4 package, just the multicast data packet is given as an example of the code.

Peer interaction protocol

We will pack all communication between peers in an envelope.

On any envelope there is always a sender and recipient, to this we add a command (which he carries with him), an identifier (as long as it is a random number, but can be made as a hash of content), the length of the content and the contents of the envelope itself - the message or command parameters.

The command, (or the type of content) is successfully located at the very beginning of the envelope and we define a list of commands of 4 bytes that do not intersect with the names of HTTP verbs.

The entire envelope, when transmitted, is serialized into an array of bytes.

Handshake

When the connection is established, the banquet immediately extends a hand for a handshake, communicating its name, public key and ephemeral public key to generate a common session key.

In response, the peer receives a similar set of data, registers the found peer in its list and calculates (CalcSharedSecret) the common session key.

  func handShake(p *proto.Proto, conn net.Conn) *proto.Peer { /* ... */ peer := proto.NewPeer(conn) /* Отправляем свое имя и ключ*/ p.SendName(peer) /* Ждем имя и ключ */ envelope, err := proto.ReadEnvelope(bufio.NewReader(conn)) /* ... */ } 

Peer Exchange

After a handshake, peers exchange their peer lists =)

To do this, an envelope is sent with the LIST command, and a JSON list of peers is put in its contents.
In response, we get a similar envelope.

We find in the lists of new ones and with each of them we make an attempt to connect, shake hands, exchange feasts and so on ...

User Message Exchange

User messages are the most valuable for us, so we will encrypt and sign each connection.

About encryption

In the standard (google) golang libraries from the crypto package, there are a lot of different algorithms implemented (no GOSTs).

I find the Ed25519 curve as the most convenient for signatures. We will use the ed25519 library to sign messages.

At the very beginning, I was thinking of using a pair of keys obtained from ed25519, not only for signing, but also for generating a session key.

However, the keys for the signature are not applicable to the calculation of the common (shared) key - they still need to be conjured over them:

 func CreateKeyExchangePair() (publicKey [32]byte, privateKey [32]byte) { pub, priv, err := ed25519.GenerateKey(nil) /* ... */ copy(publicKey[:], pub[:]) copy(privateKey[:], priv[:]) curve25519.ScalarBaseMult(&publicKey, &privateKey) /* ... */ } 

Therefore, it was decided to generate ephemeral keys, and generally speaking, this is the correct approach, which leaves no chance for attackers to find the common key.

For mathematics lovers, here are links to the wiki:
Diffie protocol —_Hellman_a_elliptic_curves
EdDSA digital signature

The generation of a shared key is quite standard: first, for a new connection, the generation of ephemeral keys, we send an envelope with a public key to the socket.

The opposite side does the same, but in a different order: it receives an envelope with a public key, generates its pair and sends the public key to the socket.

Now each participant has someone else’s public and private ephemeral keys.

Multiplying them we get the same key for both, which we will use to encrypt messages.

 //CalcSharedSecret Calculate shared secret func CalcSharedSecret(publicKey []byte, privateKey []byte) (secret [32]byte) { var pubKey [32]byte var privKey [32]byte copy(pubKey[:], publicKey[:]) copy(privKey[:], privateKey[:]) curve25519.ScalarMult(&secret, &privKey, &pubKey) return } 

We will encrypt messages by a piece by the long-proven AES algorithm in block coupling mode (CBC).

All this implementations are easily found in the golang documentation.

The only refinement is auto-filling of the message with zero bytes for the multiplicity of its length to the length of the encryption block (16 bytes).

  //Encrypt the message func Encrypt(content []byte, key []byte) []byte { padding := len(content) % aes.BlockSize if padding != 0 { repeat := bytes.Repeat([]byte("\x00"), aes.BlockSize-(padding)) content = append(content, repeat...) } /* ... */ } //Decrypt encrypted message func Decrypt(encrypted []byte, key []byte) []byte { /* ... */ encrypted = bytes.Trim(encrypted, string([]byte("\x00"))) return encrypted } 

Back in 2013, he implemented AES (with a similar CBC mode) to encrypt messages in a Telegram as part of a competition from Pavel Durov.

The most common Diffie-Hellman protocol was used in the telegrams to generate an ephemeral key at that time.

And in order to eliminate the load from fake connections, before each key exchange clients solved the factorization task.

GUI

We need to show a list of peers and a list of messages with them, as well as respond to new messages by increasing the counter next to the name of the feast.

Here without problems - ReactJS + websocket.

A web socket message is essentially a kind of envelope, only they do not contain ciphertexts.

All of them are "heirs" of the WsCmd type and are serialized during transmission to JSON.

  //Serializable interface to detect that can to serialised to json type Serializable interface { ToJson() []byte } func toJson(v interface{}) []byte { json, err := json.Marshal(v) /* обработка err */ return json } /* ... */ //WsCmd WebSocket command type WsCmd struct { Cmd string `json:"cmd"` } //WsMessage WebSocket command: new Message type WsMessage struct { WsCmd From string `json:"from"` To string `json:"to"` Content string `json:"content"` } //ToJson convert to JSON bytes func (v WsMessage) ToJson() []byte { return toJson(v) } /* ... */ 

So, HTTP request comes to root ("/"), now to display the front we look in the “front / build” directory and give it to index.html

Well, the interface is set up, now the choice for users: run it in a browser or in a separate window - WebView.

For the latter option used zserge / webview

  e := webview.Open("Peer To Peer Messenger", fmt.Sprintf("http://localhost:%v", initParams.Port), 800, 600, false) 

To build an application with it, you need to install another lib in the system.

  sudo apt install libwebkit2gtk-4.0-dev 

During my thoughts on the GUI I found a lot of libraries for GTK, QT, and the console interface would look very geeky - https://github.com/jroimartin/gocui - in my opinion a very interesting idea.

Launch messenger

Install golang

Of course, you first need to install go.
For this, I strongly recommend using the golang.org/doc/install instructions.

Simplified instructions to bash script

Download the application to GOPATH

That's the way go is, that all libraries and even your projects should be in the so-called GOPATH.

The default is $ HOME / go. Go allows you to pull the source code from the public repository with a simple command:

  go get github.com/easmith/p2p-messenger 

Now in your $HOME/go/src/github.com/easmith/p2p-messenger source from the master branch will appear

Npm installation and front assembly

As I wrote above, our GUI is a web application with a front on ReactJs, so the front still needs to be built.

Nodejs + npm - as usual here.

Just in case, here is the instruction for ubuntu

Now we run the front assembly as standard

 cd front npm update npm run build 

Front ready!

Launch

Go back to the root and run the feast of our messenger.

When starting, we can specify the name of our peer, a port, a file with addresses of other peers and a flag indicating whether to launch WebView.

By default, $USER@$HOSTNAME is used as the $USER@$HOSTNAME name and port 35035.

So, we start and chat with friends on the local network.

  go run app.go -name Snowden 

Feedback on programming on golang

• The most important thing I would like to note is: on go, it immediately turns out to realize what I had in mind .
  Almost everything you need is in the standard library.
• However, there was a difficulty when I started a project in a directory other than GOPATH.
  For writing code used GoLand. And at first, the automatic formatting of the code with the auto-import of libraries confused.
• The IDE has a lot of code generators , which allowed us to focus on development, not on typing code.
• You quickly get used to frequent error handling, but hand-to-face happens when you realize that for go there is a normal situation, when the essence of the error is analyzed by its string representation.
  

   err != io.EOF 

• Slightly better things are with the os library. They help to understand the essence of the problem.
  

   if os.IsNotExist(err) { /* ... */ } 

• Out of the box go teaches us to properly document the code and write tests.
  And there is its own but. We have described the interface with the ToJson() method.
  So, the documentation generator does not inherit the description of this method on the methods that implement it, so in order to remove unnecessary versions, you have to copy the documentation into each implemented method (proto / mtypes.go).
• Recently, I got used to the power of log4j in java, so there is not enough good logger in go.
  It is probably worth looking for a beautiful logging with appenders and formatters in the githab open spaces.
• Unusual work with arrays.
  For example, concatenation occurs through the function append , and the conversion of an array of arbitrary length into an array of fixed length through copy .
• switch-case works like an if-elseif-else - and this is an interesting approach, but again a hand-face:
  if you want the usual switch-case behavior, you need to put fallthrough on each case.
  And you can also use goto , but let's not please!
• There is no ternary operator and often this is not convenient.

What's next?

Here is the simplest Peer-To-Peer messenger.

There are bumps, you can further improve the user functionality: sending files, pictures, audio, emoticons, etc., etc.

And you can not invent your protocol, and use the google Protocol Buffers,
connect the blockchain and protect against spam using smart contracts Ethereum.

On smart contracts, organize group chats, channels, name system, avatars and user profiles.

It is also necessary to start seed peers, implement a NAT traversal and transfer messages from feast to feast.

In the end, you get a good telegram / VTS replacement, all you have to do is to transfer all friends there =)

Utility

 godoc -HTTP=:6060