How has the process of support sites for the last twenty years

Information technology guru and CTO of Ars Technica magazine, Jason Marlin, has more than twenty years of experience supporting information infrastructures — and, in his opinion, a lot has changed in this area

The Pit game, which worked as a BBS door. In this screenshot, Lee Hutchinson is attacking these guys. Or they are his.

In the 1980s, I grew up a true nerd - not in the hipster sense, but in the sense that I was carrying around with me the five-kilogram number of Computer Shopper magazine (yeah, these publications were really big). I got hooked on Bibieski (Bulletin Board Systems) by ten years. It is not surprising that in the end I turned out to be the technical director of a site covering science and technology.

I can draw clear parallels between the work of supporting my BBS (that is, the work of a sysop) and the management of a modern web infrastructure. Let us mark these parallels in honor of the 20th anniversary of the Ars website. This article will not be an exhaustive history of websites, here I will describe my own experience in website management, and how it has evolved over the last 20 years - as well as how tools and thinking have changed.

LOAD “*”, 8, 1

My first experience as a sysop was obtained on Commodore 128 (of course, in 64-bit mode), where Greg Pfountz's Color 64 program worked. I sent Greg my check (well, it was signed by my mother) and received one 5.25 "floppy disk manual instructions, printed on a matrix printer. And then it began.

Color 64 looked amazing thanks to ASCII, ANSI-colorized, unlike most other BBS programs that produced colorless, banal text. Color 64 made it possible to create a user experience. I can no longer remember the names of my BBS, but I guarantee that the main topic was related to dragons and / or kung fu. I'm a little ashamed to admit that my nickname was DragonMaster, but I only confirmed the stereotypes associated with the nyards.

Unfortunately, my network infrastructure consisted of a single telephone line, which meant that I had to disconnect all the calling devices (that is, the wall-mounted disk phone) and work between 23 and 5 hours. It also meant little BBS interactivity. With a single line and a single Commodore 1571 drive, users couldn't chat with each other or download more than one game at a time.


Commodore 1670 still makes the heart beat faster

In my dreams in the near future, I ran a real BBS, something like the then famous Las Vegas Fear & Loathing, which I often hover for. In my dreams, there were 10 telephone lines, so that users could communicate with each other in real time, and connect to 1200 modems — no, not even 2400 baud! And on a mythical hard disk of as much as 10 MB of system Lt. Kernal would have stored endless stockpiles of games.

Alas, it was all inaccessible to me, but I clearly contracted some new disease, which encouraged an unusual desire in a person to create digital places in which users could gather

1990s

I continued to tinker with the software for the BBS, including with such a very interesting predecessor of HTML, as Excalibur BBS . Take a look at the image search results on Google to see how much this software is ahead of its time.

$: cd ~ / public_html

I first met HTML in college in the mid-90s; then I did my homework and uploaded them to the public home directory, and teachers could view them in their free time using Netscape or Mosaic browsers. An excellent motivator then was an additional 10 points for "using technology."

Apache + Perl + XML + Shared Hosting

One of the first real “applications” that I created as a web developer was the news section for a telecommunications company. All this worked on a common link: Apache as an HTTP server, Perl as a server language, and a database as a text file. At that time I was not familiar with real databases, but I knew how to write and parse XML. All this was hosted on a surprisingly good joint platform, on which I could write all the rules for the server to work in the .htaccess file. Soon I learned that this file gave my inexperienced hands too much power!

My tasks shared hosting solved, but at that time the developers were dependent on admins in everything related to software versions and extensions. You also had to worry about what your neighbors are doing with shared resources, including all sorts of nasty things. Hacking one machine could compromise hundreds of sites.

IIS, FrontPage Extensions and Access

As a result, the agency where I worked gained enough customers to afford their own server. To my chagrin, this turned out to be a Windows machine running IIS (Internet Information Services). This territory was completely unfamiliar to me, but after launching the Frontpage IDE, I was amazed at how simple Microsoft usually did the difficult task of storing valid input data into the database. (No, seriously, just awesome). As a result, I went in search of the perfect graphic IDE, including a brief and disappointing romp with Macromedia Dreamweaver. Soon I learned that the tools that create the code automatically usually give out a huge amount of noodles, which only the same tools could unravel.

Managing IIS under Windows NT 3.5 also looked like a dangerous, easy task for a person with Unix experience. And at the same time it was a feeling of tight restrictions - where were the .conf files in which it was possible to do micro-tuning (and phenomenally fake)?

This build has become my platform for some time, we have made several CMSs for our customers to order, without any thoughts about maintaining a common code base, long-term support or version control. Here is a horror thing.

Early 2000s Beginning with ColdFusion

Now I understand the horror of the situation, but then I really liked the Allaire ColdFusion environment, and I used it for at least four years to create fairly large applications and corporate networks. She worked on the basis of CFML (ColdFusion markup language). It was similar to HTML, but it trivially queried databases and integrated them with external technologies like Java Servlets or CORBA. ColdFusion had many opponents, but I was never a supporter of any particular technology, and simply chose the one that allowed me to complete the task most quickly.

Web frameworks appear

ColdFusion's very low entry barrier has earned it the ill fame of the idiot language, which has attracted low-quality programmers to this field - just like PHP used to do. I can not argue with this, but the irony is that my first acquaintance with a good web framework took place in the form of Fusebox . It began with an attempt to organize an application using simple file naming and directory system conventions. It sounded obvious, but, like most web developers at the time, I was struggling with an ever-evolving approach to the layout of the application and struggled with such detachable things as database queries and output components. I played with Struts , but since it was impossible to use Java in my main job, I didn’t comprehend it. But Fusebox opened my eyes to platforms with the MVC paradigm (model view controller) that transcends specific languages. And it was many years before the appearance of that 15-minute Ruby on Rails demolishing roof, made by David Heinemeyer Hansson .

Today, I would never consider creating a large application without choosing a framework, and today we have many interesting choices. For PHP, I like Laravel the most.



Yes, then it just blew off the roof.

Cluster web hosting

My first experience with the high traffic website was in 2002. With increasing traffic, responsibility increased, as well as the number of calls in the middle of the night demanding to restart the server. And finally, I decided to find out everything about load balancing, caching and cluster hosting. This was another revelation, because it opened up the possibility of almost unlimited scaling.

If one car went offline, we had safety cars to ensure that everything worked. We had analytics and a detailed metric on a cluster. Life was beautiful, as was my dream.

Rise of the Virtual Machines: AWS, Vagrant

AWS (Amazon Web Services) appeared, seemingly out of nowhere, and gave the developers exactly what we needed. They also eliminated intermediaries from traditional hosting. We did not need to ask which technologies we were allowed to use; suddenly everything was possible. Want to try making an application on Django or NodeJS? No problem. Run a couple of virtual locks and go ahead. All this could be done with AWS: virtual firewalls, load balancers, special clusters for databases, CDN (content delivery network) for static resources, and almost everything that could be invented. It became a data center for self-production - and it was both a curse and a blessing. When adding each new service, it was necessary to track it, and so that someone knew how to pick it up after everything breaks (and everything breaks). Too enthusiastic developer was very easy to bite off more than he could chew.

What AWS made possible in the cloud, Vagrant made it possible on a working machine. Vagrant gave us easy and scriptable access to various VM providers. I could test new types of Linux and all sorts of software packages in the environment, which, when the moment of deployment came, it was very easy to recreate in the cloud. If something went wrong during the installation, the simplest vagrant destroy command let you start over from the beginning with the same image. This made development much more enjoyable than running servers on a home OS, or directly using VMWare or VirtualBox.

2010s - from webmasters to DevOps

Let's slow down a bit and talk about how I always hated the term webmaster.

Man: What are you doing in life?
I: create funny web applications using various programming languages, and work with the infrastructure ...
Man: Ahhh, you're a WEBMASTER!

I want to remove this disgusting word from use.

Besides the fact that this word is associatively associated with a 20-sided game dice, it does not at all describe the role that many of us play in programming and managing infrastructure. So I like this shift over the last decade so much, towards more respected terms like DevOps . DevOps engineers use programming to create, manage, and document real-world infrastructure.

Chef + ansible

When I started working at Ars Technica, we wanted to be able to easily add or remove web servers and servers with databases in a virtualized environment. After examining several approaches, we stopped at Chef , which simplifies infrastructure management through a hierarchy of analogies, mainly related to the kitchen (knife, recipe book, recipes, etc.). Having studied how the properties of variables, or “attributes,” as they are called here, flow from roles and are reduced to individual nodes, it becomes very easy to manage server software and versions from a single platform. Chef allowed us to stop working on the micromanagement of individual servers in clusters and eased the task of mass upgrades.

Today, personally, I prefer to work with Ansible , a system based on Python, and developed in Redhat - it is easier to work with it and it is a little less fragile when working in small organizations. Unlike Chef, where a central server is required, Ansible uses SSH from a management machine (in our case, from our laptops with which we are developing), but for larger enterprises it still has a server, Tower. Ansible also allows you to write most of the configuration to YAML, which greatly improves its readability.

For several years now we have hosted a ServerCentral Turing Group . They help us choose the right balance between what we are good at (writing code and setting up VM) and what we can’t fully manage - safety diesel generators, redundant networks or real-time copying of data into a data center intended to bypass failures.

2019 and further

I will always feel nostalgia for those serene days of manually entering commands for a modem, when somewhere on the horizon we were beckoned by promises made by the Lawn Mower or Hal 9000. But the current moment also contains tremendous possibilities of the next phase of infrastructure evolution. There are still so many promising tools that we plan to use, for example, Docker Swarm or Kubernetes . It's amazing to know how far we have gone, and how much more productive a today's developer can be compared to those old days filled with the mysterious squeaking of BBS. I expect that we will be watching the ever-increasing abstraction of complex multi-layered technology necessary for a modern web presence.

So we wish you success in the next 20 years!