Produced | Open Source China
The author | boiled water without added sugar
InfoWorld has published a list of the best open source software of 2021.
InfoWorld is an international technology media brand dedicated to leading IT decision makers at the forefront of technology, and each year InfoWorld selects the "Best of Open Source Software Awards, Bossies" of the year based on the contribution of software to the open source community and its influence in the industry, which has been awarded for more than a decade.
InfoWorld said the 28 open source projects that won the award represent the best and most innovative software that open source software can offer today — software development, development, cloud-native computing, machine learning, and more.
Next, take a closer look at each project (click on each project name to jump to the project introduction page).
Svelte and SvelteKit
InfoWorld reviews that of the many innovative, open source, front-end JavaScript frameworks, Svelte and its full-stack counterpart, SvelteKit, are probably the most ambitious and visionary. Svelte started by adopting a compile-time strategy to disrupt the status quo and move forward with great performance, continuous growth, and a great developer experience. SvelteKit, now in beta, continues Svelte's tradition of making the leap forward by adopting the latest tools and deploying to serverless environments as a built-in feature.
Minikube
InfoWorld believes that Minikube could be considered as an alternative to Docker Desktop. Minikube is an easy-to-run Kubernetes tool that makes it easy to create stand-alone Kubernetes clusters in virtual machines on your laptop. Easy to try Kubernetes or use Kubernetes for everyday development.
Pixie
Pixie is an observability tool for Kubernetes apps that allows you to view the high-level status of your cluster, such as a service map, cluster resources, and application traffic, and to drill down into more detailed views such as pod status, flame maps, and individual full-body app requests. Pixie uses eBPF to automatically collect telemetry, which collects, stores, and queries all telemetry locally in the cluster, using less than 5% of the cluster CPU. Use cases for Pixie include network monitoring within a cluster, infrastructure health, service performance, and database query profiling.
FastAPI
FastAPI is a high-performance web framework for building APIs. Key features:
Fast: Very high performance, comparable to NodeJS and Go
Fast coding: Accelerate feature development by approximately 200% to 300%
Fewer errors: Reduce human errors by about 40%.
Intuitive: Powerful editor support, auto-completion is everywhere, and debugging time is less
Simplicity: Designed to be easy to use and learn, reducing the time spent reading documents.
Short: Reduce code duplication.
Robust: Get code available for production with automated interactive documentation
Standards-based: Open API and JSON Schema, an open standard based on and fully compatible APIs
Crystal
Crystal has been developed for several years as a project that offers a programming language that has the speed of C and the expressive power of ruby. With the release of Crystal 1.0 earlier this year, the language is now stable enough to be used for general workloads. Crystal uses static typing and the LLVM compiler to achieve high speed and avoid common problems such as null references at runtime. Crystal interfaces with existing C code for further speed and convenience, and it can also extend the syntax of the underlying language using compile-time macros.
Windows Terminal
Windows Terminal is a new, popular, and powerful command-line terminal tool. Includes many features that are highly spoken out of the community, such as: multi-tab support, rich text, multi-language support, configurable, themes and styles, support for emoji and GPU-based text rendering, and so on. At the same time, the terminal still meets our goals and requirements to ensure that it remains fast, efficient, and does not consume a lot of memory and power.
InfoWorld says that given time, Windows Terminal will one day replace the old console host in Windows.
OBS Studio
OBS Studio is a software for live streaming and screen recording designed to efficiently capture, synthesize, encode, record and stream video content, supporting all streaming platforms.
characteristic:
High-performance real-time video/audio capture and mixing. Create scenes from multiple sources, including window captures, images, text, browser windows, webcams, capture cards, and more.
Set an unlimited number of scenes, users can seamlessly switch through custom transitions.
Intuitive audio mixers with each source filter, such as noise gate, noise suppression, and gain. Full control over VST plug-in support.
Powerful and easy-to-use configuration options. Add new sources, copy existing ones, and easily adjust their properties.
The streamlined settings panel gives users access to a variety of configuration options to adjust aspects of broadcasting or recording.
The modular "Dock" UI allows users to rearrange the layout exactly as needed. Users can even pop each individual Dock into their own window.
Shotcut
Shotcut is a cross-platform video editing tool that allows people to make all standard corrections to audio and video tracks while applying effects and layering. Shotcut has a very active community and offers plenty of how-to videos and instructions to help novice and advanced videographers. It can run on Mac, Linux, BSD, and Windows — although cross-platform, it's agile and relatively simple to use compared to similar tools.
Weave GitOps Core
Weave GitOps supports effective GitOps workflows to continuously deliver applications to Kubernetes clusters. It is based on the leading GitOps engine, CNCF Flux.
Apache Solr
Apache Solr is a Lucene-based full-text search server and the most popular enterprise search engine. Apache Lucene is the basic search technology behind the search capabilities of most of the software you use – including other search engines, such as Elasticsearch. Unlike Elasticsearch, Solr abandoned its open source license, but it's still free. Solr is clusterable, deployable in the cloud, and powerful enough to build a cloud-class search service. It even includes LTR algorithms to help automatically adjust and weight results.
MLflow
Created by Databricks and hosted by the Linux Foundation, MLflow is an MLOps platform that enables people to track, manage, and maintain a variety of machine learning models, experiments, and their deployments. It gives you the tools to document and query experiments (code, data, configuration, results), package data science code into projects, and chain those projects into workflows.
Orange
Orange aims to make data mining "productive and fun". Orange allows users to create a data analysis workflow, perform various machine learning and analysis functions, and visualize. Compared to procedural or textual tools like R Studio and Jupyter, Orange is very intuitive. You can drag widgets onto the canvas to load files, analyze data with models, and visualize the results.
Flutter
Flutter was built by Google's team of engineers to create high-performance, cross-platform mobile apps. Optimized for today's and tomorrow's mobile devices, Flutter focuses on low-latency input and high frame rates for Android and iOS.
Flutter provides developers with a simple and efficient way to build and deploy cross-platform, high-performance mobile apps, and provides users with a beautiful, fast, jitter-free app experience.
Apache Superset
Apache Superset is An open source data exploration and visualization platform (formerly known as Panoramix, Caravel) by Airbnb (a well-known online housing short-term rental company), which is very distinctive in terms of visualization, ease of use and interactivity, allowing users to easily visualize and analyze data. Apache Superset is also an enterprise-grade business intelligence web application.
Soon
Presto is an open source distributed SQL engine for online analytics processing that runs in clusters. Presto can query a wide variety of data sources, from files to databases, and return the results to many business intelligence and analytics environments. What's more, Presto allows querying where the data resides, including Hive, Cassandra, relational databases, and proprietary data stores. A Presto query can combine data from multiple sources. Facebook uses Presto to make interactive queries about several internal data stores, including their 300PB data warehouse.
Facebook, Uber, Twitter and Alibaba founded the Presto Foundation. Other members now include Alluxio, Ahana, Upsolver and Intel.
Apache Arrow
Apache Arrow defines a language-independent columnar memory format for flat and hierarchical data, organized for efficient analysis operations on modern CPUs and GPUs. The Arrow memory format also supports zero-copy reads for lightning-fast data access without serialization overhead. The Arrow library is available in C, C++, C#, Go, Java, JavaScript, Julia, MATLAB, Python, R, Ruby, and Rust.
InterpretML
InterpretML is an open source Explainable AI (XAI) package that contains several of the most advanced machine learning interpretability techniques. InterpretML lets you train interpretable glassbox models and interpret black box systems. InterpretML can help you understand the global behavior of your model, or understand the reasons behind individual predictions. Among its many features, InterpretML has a "glass box" model from Microsoft Research called explainable Boosting Machine, which supports Lime with post-hoc interpretations of black-box models.
Lime
Lime (short for local interpretable model-agnostic explanations) is a post-hoc technique that interprets the predictions of any machine learning classifier by perturbing the characteristics of the input and examining the prediction results. Lime is able to interpret any black box classifier with two or more classes, which applies to both the text and image fields. Lime is also included in InterpretML.
Dask
Dask is an open source library for parallel computing that extends Python packages to multiple machines. Dasks can distribute data and compute across multiple GPUs, whether in the same system or in a multi-node cluster. Dask integrates with Rapids cuDF, XGBoost, and Rapids cuML for GPU-accelerated data analytics and machine learning. It also integrates with NumPy, Pandas, and Scikit-learn to parallelize its workflows.
BlazingSQL
BlazingSQL is a GPU-accelerated SQL engine built on the RAPIDS ecosystem. BASED ON THE APACHE ARROW COLUMN MEMORY FORMAT, RAPIDS IS A GPU DATAFRAME LIBRARY FOR LOADING, CONNECTING, AGGREGATING, FILTERING, AND MANIPULATING DATA.
BlazingSQL is cuDF's SQL interface with various features to support large-scale data science workflows and enterprise datasets.
Rapids
Nvidia's Rapids open source software libraries and API suite give you the ability to perform end-to-end data science and analytics pipelines entirely on the GPU. Rapids uses Nvidia CUDA primitives for low-level computational optimization and exposes GPU parallelism and high-bandwidth memory speeds through a user-friendly Python interface. Rapids relies on the Apache Arrow column memory format, including cuDF, a Pandas-like DataFrame library; cuML, a collection of machine learning libraries that provide GPU versions of most of the algorithms in Scikit-learn; and cuGraph, a NetworkX-like accelerated graph analysis library.
PostHog
PostHog is an open source product analytics platform built for developers. Automatically collect every incident on your website or app without sending data to third parties. It provides event-based analytics at the user level, capturing usage data for your product to see which users are doing what in your application. It automatically captures clicks and combined views to analyze what your users are doing without having to manually push events.
LakeFS
LakeFS provides a "manage your data lake in the same way you manage your code", adding a layer of Git-like versioning to object storage. This application of Git semantics allows users to create their own isolated, zero-copy data branches to work on, experiment, and model without the risk of breaking shared objects. LakeFS brings useful commit notes, metadata fields, and rollback options to your data, as well as validation hooks that maintain data integrity and quality—running format and schema checks before an uncommitted branch is accidentally merged back into production. With LakeFS, the familiar techniques for managing and securing your codebase can be extended to modern databases such as Amazon S3 and Azure Blob storage.
Meltano
Meltano was spun off from GitLab this year, a free and open source DataOps toolchain that replaces traditional ELT (extract, load, convert). Meltano's data warehouse framework makes it easy to model, extract, and transform data for your projects, complementing the integration and transformation pipeline with built-in analytics tools and dashboards that simplify reporting. With a robust library of extractors and loaders, as well as support for Singer's standard data extracting taps and data loading targets, Meltano is already a power source for data orchestration.
Triune
Trino (formerly known as PrestoSQL) is a distributed SQL analysis engine capable of running extremely fast queries against large distributed data sources. Trino allows you to execute queries on data lakes, relational storage, or multiple different sources at the same time without having to copy or move data for processing. And Trino works well with any business intelligence and analytics tools your data scientist might use, whether interactive or ad hoc, minimizing the learning curve. As data engineers strive to support complex analysis of more and more data sources, Trino provides a way to optimize query execution and accelerate results from different sources.
StreamNative
StreamNative is a highly scalable message and event streaming platform that greatly simplifies the laying of data pipelines for real-time reporting and analytics tools and enterprise application streams. StreamNative combines Apache Pulsar's powerful distributed stream processing architecture with enterprise extras such as Kubernetes and hybrid cloud support, a large database of data connectors, easy authentication and authorization, and dedicated tools for health and performance monitoring, simplifying the development of real-time Pulsar-based applications and the deployment and management of large-scale messaging backplanes.
Hugging Face
Hugging Face provides the most important open source deep learning repository, which is not a deep learning framework in itself. The goal of Hugging Face is to expand beyond text, supporting images, audio, video, object detection, and more. Infoworld notes that deep learning practitioners should pay close attention to this repo for years to come.
EleutherAI
EleutherAI is a distributed group of machine learning researchers designed to bring GPT-3 to everyone. At the beginning of 2021, EleutherAI released The Pile, an 825 GB dataset of diverse text for training, and in June unveiled GPT-J, a 6 billion-parameter model roughly equivalent to OpenAI's Curie variant of GPT-3. With the advent of GPT-NeoX, EleutherAI plans to raise the parameter all the way to 175 billion to compete with the broadest GPT-3 model available.
In response, Infoworld commented, "Hackers vs. the world's largest companies? That's the power of open source. ”
Colab notebooks for generative art
Infoworld says that in general, the winners of Bossies are the backbone of open source such as libraries, frameworks, platforms and operating systems. This year, however, they believe that some of the great open source components should also be recognized.
The first is OpenAI's CLIP (Contrast Language- Image Pre-Training) model, a multimodal model for generating vector embeddings of text and images. While CLIP is completely open source, OpenAI's generative neural network, DALL-E, is not. To fill this gap, Ryan Murdoch and Katherine Crowson developed Colab notebooks, combining CLIP with other open source models such as BigGAN and VQGAN to produce prompt-based generative artwork. These notebooks, based on the MIT license, have been widely disseminated across the Internet over the past few decades, remixed, altered, translated, and used to produce amazing works of art. See ai_curio for details.
The above is the 2021 InfoWorld Bossie Awards project, and details such as the specific selected comments for each project can be found in the original website.
Related links: https://www.infoworld.com/article/3637038/the-best-open-source-software-of-2021.html#slide1