📜 ⬆️ ⬇️

xenvman: Flexible environments for testing microservices (and not only)

Hello!


I would like to tell you a little about the project I have been working on for the past six months. I do the project in my spare time, but the motivation for its creation came from observations made at the main work.


On the working draft, we use the microservice architecture, and one of the main problems that has manifested itself over time and with the increased number of these same services is testing. When a service depends on five to seven other services, plus some other database (or even a few) in addition, it is very inconvenient to test it in a “live” form, so to speak. It is necessary to be surrounded by mocks from all sides so tightly that the test itself cannot be seen. Well, or somehow organize the test environment, where all the dependencies could actually be running.


Actually to facilitate the second option, I just sat down to write xenvman . In a nutshell, it is something like a hybrid of docker-compose and test containers , only without binding to Java (or any other language) and with the ability to dynamically create and configure environments via the HTTP API.


xenvman written on Go and is implemented as a simple HTTP server, which allows you to use all the available functionality from any language that can speak this protocol.


The main thing that xenvman can do is:



Environments


The main protagonist in xenvman is the environment. This is such an isolated bubble in which all the necessary dependencies (packed in Docker containers) of your service are launched.



The figure above shows the xenvman server and active environments where various services and databases are running. Each environment was created directly from the integration test code, and will be deleted upon completion.


Templates


What is directly included in the environment is determined by the templates (templates), which are small scripts in JS. xenvman has a built-in interpreter for this language, and when it receives a request to create a new environment, it simply executes the specified templates, each of which adds one or more containers to the list for execution.


JavaScript was chosen to allow dynamically change / add templates without having to rebuild the server. In addition, as a rule, only basic features and data types of the language are used in the templates (good old ES5, no DOM, React and other magic), so working with templates should not cause any particular difficulties even for those who do not know JS.


Templates are parameterizable, that is, we can completely control the template logic by passing certain parameters in our HTTP request.


Creating images on the fly


One of the most convenient features of xenvman, in my opinion, is the creation of Docker images right along with the configuration of the environment. Why it may be necessary?
Well, for example, we have on a project to get an image of a service, we need to commit changes into a separate branch, push and wait while Gitlab CI collects and floods the image.
If only one service has changed, then it can take 3-5 minutes.


And if we are actively sawing new features in our service, or are trying to understand why it does not work by adding the good old fmt.Printf to and fro, or changing the code some other way, even a delay of 5 minutes will be great to put out the performance ( ours, as code writers). Instead, we can simply add all the necessary debugging to the code, compile it locally, and then simply attach the finished binary to the HTTP request.


Having received such good, the template will take this binar and on the move will create a temporary image from which we can already launch the container as if nothing had happened.


On our project, in the main template for services, for example, we check if the binary is present in the parameters, and if so, then collect the image on the go, otherwise just download the latest version of the dev branch. The following code for creating containers is identical for both options.


Small example


For clarity, let's look at the micro-sample.


Let's say we write some kind of miracle server (let's call it wut ), which needs a database to store everything there. Well, as a base, we chose MongoDB. So for full testing, we need a working Mongo server. You can, of course, install and run it locally, but for simplicity and clarity of the example, we assume that for some reason this is difficult to do (with other, more complex configurations in real systems, this will be more like the truth).


So we will try to use xenvman to create an environment with Mongo running and our wut server.


First we need to create a base directory in which all templates will be stored:


$ mkdir xenv-templates && cd xenv-templates


Next, create two templates, one for Mongo, the other for our server:


$ touch mongo.tpl.js wut.tpl.js


mongo.tpl.js


Open mongo.tpl.js and write the following there:


 function execute(tpl, params) { var img = tpl.FetchImage(fmt("mongo:%s", params.tag)); var cont = img.NewContainer("mongo"); cont.SetLabel("mongo", "true"); cont.SetPorts(27017); cont.AddReadinessCheck("net", { "protocol": "tcp", "address": '{{.ExternalAddress}}:{{.Self.ExposedPort 27017}}' }); }

The execute () function with two parameters must be present in the template.
The first is the tpl instance of the object through which the environment is configured. The second argument (params) is just a JSON object, with which we will parameterize our template.


In line


 var img = tpl.FetchImage(fmt("mongo:%s", params.tag));

we ask xenvman to download the docker mongo:<tag> image mongo:<tag> , where <tag> is the version of the image we want to use. In principle, this line is equivalent to the docker pull mongo:<tag> , with the only difference that all the functions of the tpl object are essentially declarative, that is, the image will actually be downloaded only after xenvman executes all the templates specified in the environment configuration.


Once we have an image, we can create a container:


 var cont = img.NewContainer("mongo");

Again, the container will not be instantly created in this place, we simply declare the intention to create it, so to speak.


Next, we label our container:


 cont.SetLabel("mongo", "true");

Labels are used so that containers can find each other in the environment, for example, to enter the IP address or host name in the configuration file.


Now we need to hang the inner Mongo port (27017) out. This is easily done like this:


  cont.SetPorts(27017);

Before xenvman reports on the successful creation of the environment, it would be great to make sure that all services are not just running, but already ready to accept requests. Xenvman has readiness checks for this.
Add one for our mongo container:


  cont.AddReadinessCheck("net", { "protocol": "tcp", "address": '{{.ExternalAddress}}:{{.Self.ExposedPort 27017}}' });

As we can see, here in the address bar there are stubs into which the necessary values ​​will be dynamically substituted just before launching the containers.


Instead of {{.ExternalAddress}} external address of the host on which xenvman is running will be substituted, and instead of {{.Self.ExposedPort 27017}} an external port will be substituted, which was forwarded to the internal 27017.


Read more about interpolation here .


As a result of all this, we will be able to connect to Mongo, running in the environment, right outside, for example, from the host on which we run our test.


wut.tpl.js


So-c, having understood Monga, we will write another template for our wut server.
Since we want to build the image on the go, the pattern will be slightly different:


 function execute(tpl, params) { var img = tpl.BuildImage("wut-image"); img.CopyDataToWorkspace("Dockerfile"); // Extract server binary var bin = type.FromBase64("binary", params.binary); img.AddFileToWorkspace("wut", bin, 0755); // Create container var cont = img.NewContainer("wut"); cont.MountData("config.toml", "/config.toml", {"interpolate": true}); cont.SetPorts(params.port); cont.AddReadinessCheck("http", { "url": fmt('http://{{.ExternalAddress}}:{{.Self.ExposedPort %v}}/', params.port), "codes": [200] }); }

Since we are BuildImage() image here, we use BuildImage() instead of FetchImage() :


  var img = tpl.BuildImage("wut-image");

In order to build an image, we will need several files:
Dockerfile - the actual instruction how to build the image
config.toml - configuration file for our wut server


Using the img.CopyDataToWorkspace("Dockerfile"); method img.CopyDataToWorkspace("Dockerfile"); we copy the Dockerfile from the template data directory to a temporary working directory .


The data directory is the directory in which we can store all the files our template needs in the work.


In the temporary working directory, we copy the files (using img.CopyDataToWorkspace ()), which will fall into the image.


Then follows this:


  // Extract server binary var bin = type.FromBase64("binary", params.binary); img.AddFileToWorkspace("wut", bin, 0755);

We pass our server's binary directly in the parameters, encoded (base64). And in the template we simply decode it, and save the resulting string to the working directory as a file under the name wut .


Then we create a container and mount the configuration file into it:


  var cont = img.NewContainer("wut"); cont.MountData("config.toml", "/config.toml", {"interpolate": true});

A call to MountData() means that the config.toml file from the data directory will be mounted inside the container as /config.toml . The interpolate flag instructs the xenvman server to replace all existing stubs before mounting the file.


Here is what the config might look like:


 {{with .ContainerWithLabel "mongo" "" -}} mongo = "{{.Hostname}}/wut" {{- end}}

Here we are looking for a container labeled mongo , and substitute the name of its host, whatever it is in this environment.


After substitution, the file may look like:


 mongo = “mongo.0.mongo.xenv/wut”

Then we post the port again and start the readiness check, this time HTTP:


 cont.SetPorts(params.port); cont.AddReadinessCheck("http", { "url": fmt('http://{{.ExternalAddress}}:{{.Self.ExposedPort %v}}/', params.port), "codes": [200] });

On this, our templates are ready, and we can use them in the code of integration tests:


 import "github.com/syhpoon/xenvman/pkg/client" import "github.com/syhpoon/xenvman/pkg/def" // Создаём xenvman клиент cl := client.New(client.Params{}) // Требуем создать для нас окружение env := cl.MustCreateEnv(&def.InputEnv{ Name: "wut-test", Description: "Testing Wut", // Указываем, какие шаблоны добавить в окружение Templates: []*def.Tpl{ { Tpl: "wut", Parameters: def.TplParams{ "binary": client.FileToBase64("wut"), "port": 5555, }, }, { Tpl: "mongo", Parameters: def.TplParams{"tag": “latest”}, }, }, }) // Завершить окружение после окончания теста defer env.Terminate() // Получаем данные по нашему wut контейнеру wutCont, err := env.GetContainer("wut", 0, "wut") require.Nil(t, err) // Тоже самое для монго контейнера mongoCont, err := env.GetContainer("mongo", 0, "mongo") require.Nil(t, err) // Теперь формируем адреса wutUrl := fmt.Sprintf("http://%s:%d/v1/wut/", env.ExternalAddress, wutCont.Ports[“5555”]) mongoUrl := fmt.Sprintf("%s:%d/wut", env.ExternalAddress, mongoCont.Ports["27017"]) // Всё! Теперь мы можем использовать эти адреса, что подключиться к данным сервисам из нашего теста и делать с ними, что захочется

It may seem that writing templates will take too much time.
However, with the right design, this is a one-time task, and then the same templates can be re-used again and again (and even for different languages!) Simply by fine-tuning them by passing certain parameters. As you can see in the example above, the test code itself is very simple, due to the fact that all the husks for setting up the environment we have taken out in the templates.


In this small example, not all the features of xenvman are shown, a more detailed step-by-step tutorial is available here.


Customers


At the moment there are clients for two languages:


Go
Python


But adding new ones is not difficult, since the provided API is very, very simple.


Web interface


In version 2.0.0, a simple web interface was added, with which you can manage environments and view available templates.





How is xenvman different from docker-compose?


Of course, there are many similarities, but xenvman seems to me a bit more flexible and dynamic approach, compared to the static configuration in the file.
Here are the main distinguishing features, in my opinion:



Links


Github project page
Detailed step by step example, in English.


Conclusion


That's all. In the near future I plan to add the opportunity
call templates from templates and thus allow to combine them with greater efficiency.


I will try to answer any questions, and I will be glad if this project turns out to be useful to anyone else.



Source: https://habr.com/ru/post/439236/

More articles: