Month: March 2024

Article originally posted on InfoQ. Visit InfoQ

Recently, GitHub talked about using merge queues to implement code updates. Merge queues have been developed and scaled to manage over 30,000 pull requests, alongside the corresponding 4.5 million CI executions, for GitHub.com.

The merge queue system organizes pull requests into deployable batches, initiates builds and tests through GitHub Actions, and maintains the integrity of the main branch by preventing updates with failing commits, by following branch protection regulations. Conflicting pull requests within the queue are automatically identified and excluded, with the system regrouping as necessary.

Will Smythe, staff product manager, and Lawrence Gripper, staff engineer at GitHub, elaborated on GitHub’s journey with merge queues in a blog post. Aside from increasing developer velocity, the move to merge queues was aimed at enhancing the overall experience for developers in shipping their work, preventing problematic pull requests from affecting the broader team, and ensuring a consistent and highly automated process across all services and repositories.

By mid-2021, GitHub began piloting the merge queue feature with several smaller, internal repositories. They implemented changes to the process in stages, which involved testing and reverting modifications early in the morning before the majority of developers started their workday. Over time, GitHub systematically migrated its large monorepo and all repositories linked to production services to the merge queue system by 2023.

The merge queue blends into GitHub’s existing pull request workflow, eliminating the need for developers to learn specific ChatOps commands, or to utilize labels or particular comment syntax for state management, enhancing GitHub’s developer experience. Developers can easily queue their pull requests and, if they identify any issues with their changes, exit the queue with one click.

Following the general release of the merge queue by GitHub in around Q3 2023, we stumbled upon a relevant discussion on Hacker News. The tech community participated actively in the discussion, with a user highlighting their few months-long uses of the system for monorepo pull request merges, acknowledging its substantial improvements to the process. They also praised the system’s faster ability and reliable trunk-based releases, expressing appreciation towards the team behind the feature.

Another user inquired whether merge queues would be arriving in Azure DevOps anytime soon, indicating interest in its availability. One participant responded in the thread perceiving the lack of updates in Azure Repos and pointing out its continued reliance on ssh-rsa host keys for SSH Git—a protocol deprecated by OpenSSH for several years.

Monthly, more than 500 engineers utilize merge queue to integrate 2,500 pull requests into GitHub’s large monorepo, resulting in a 33% reduction in the average time to deploy a change. In a periodic developer satisfaction survey conducted by GitHub, an engineer applauded the merge queue as “one of the most significant quality-of-life enhancements for deploying changes that I’ve witnessed at GitHub!”

Article originally posted on InfoQ. Visit InfoQ

Databricks launched DBRX, a new open-source large language model (LLM) that aims to redefine the standards of open models and outperform well-known competitors on industry benchmarks.

With 132 billion parameters, DBRX has demonstrated in its own run of industry benchmarks that this model outperforms popular open-source LLMs such as LLaMA 2 70B, Mixtral, and Grok-1 across various language understanding, programming, and math tasks. The new model even competes favorably against Anthropic’s closed-source model Claude on specific benchmarks.

The AI Community expressed excitement about the release of DBRX, with Clem Delangue, CEO at Hugging Face posting on X:

Not a surprise but DBRX is already #1 trending on HF!

DBRX’s performance is attributed to its more efficient mixture-of-experts architecture, making it up to 2x faster at inference than LLaMA 2 70B despite having fewer active parameters. Databricks claims that training the model was also approximately 2x more compute-efficient than dense alternatives.

The model was pretrained on 12 trillion tokens of curated text and code data, leveraging advanced technologies like rotary position encodings and curriculum learning during pretraining. Developers can interact with DBRX via APIs or utilize Databricks’ tools to fine-tune the model on their proprietary data. Integration into Databricks’ AI products is already underway. DBRX is available on GitHub and Hugging Face.

Databricks anticipates that developers will adopt the model as a foundation for their own LLMs, potentially enhancing customer chatbots or internal question-answering systems. This approach also provides insight into how DBRX was constructed using Databricks’ proprietary tools.

To create the dataset utilized in developing DBRX, Databricks utilized Apache Spark and Databricks Notebooks for data processing, Unity Catalog for data management and governance, and MLflow for experiment tracking.

DBRX sets a new standard for open-source AI models, offering customizable and transparent generative AI solutions for enterprises. A recent survey from Andreessen Horowitz, indicates a growing interest among AI leaders to increase open-source adoption as fine-tuned models approach closed-source performance levels. Databricks expects DBRX to accelerate the shift from closed to open-source solutions.

Article originally posted on InfoQ. Visit InfoQ

The creators of DBOS have recently introduced DBOS Cloud, a transactional serverless application platform tailored for TypeScript developers. With all state information stored in a highly available DBMS, this new platform assures transactional serverless computing, offering reliable execution alongside so-called “time travel” capabilities.

Dubbed as “the world’s first cloud-native operating system” and a “database alternative to Kubernetes”, DBOS (DataBase oriented Operating System) implements operating system services in SQL, operating atop a high-performance distributed, transactional, partitioned fault-tolerant database. Michael Stonebraker, computer scientist and a Turing Award laureate, writes:

The idea for DBOS (DataBase oriented Operating System) originated 3 years ago with my realization that the state an operating system must maintain (files, processes, threads, messages, etc.) has increased in size by about 6 orders of magnitude since I began using Unix on a PDP-11/40 in 1973. As such, storing the OS state is a database problem. Also, Linux is legacy code at the present time and is having difficulty making forward progress. For example, there is no multi-node version of Linux, requiring people to run an orchestrator such as Kubernetes.

According to the authors, DBOS Cloud automatically logs every step an application takes and each change it makes in the database. Two distinct features of DBOS Cloud are reliable execution and time travel: in the event of an interruption to the code running on a DBOS program, it automatically resumes from the point of interruption without re-executing any previously completed work. Stonebraker adds:

Providing such guarantees yourself is months of work, but in DBOS, they’re built into every program (…) You can step through past executions to reproduce rare bugs and even run new code against historical state.

Furthermore, a “time travel debugger” enables developers to replay any DBOS Cloud trace locally on a laptop, observe past application states, and test code changes. In the future, the team plans to release a time travel functionality for disaster recovery, enabling developers to roll back an application and its data to any previous state. Jeremy Daly, CEO and founder of Ampt, comments:

What makes this super interesting is that the project was founded by Dr. Mike Stonebraker (he created Ingress, PostgreSQL, and VoltDB) and Matei Zaharia, the creator of Apache Spark.

Peter Zaitsev, founder at Percona and open source advocate, agrees but adds:

Mike Stonebraker does not seem to be a big fan of Open Source for his companies in recent years.

While the team has released an open-source DBOS TypeScript SDK, unlike Ingress, PostgreSQL, VoltDB, or Apache Spark, DBOS itself is not open-source. In a thread on Hacker News, Peter Kraft, co-founder at DBOS, explains:

Under the hood of our cloud platform (mostly Go + SQL), we’re building on ideas from the academic project to provide new features like reliable execution/time travel, but like a good OS, we want to hide that complexity from users.

A free tier and a programming guide covering idempotency and workflow execution on the serverless computing platform are now available. The free tier offers fixed resources per application (a Firecracker microVM with 512 MB of RAM and 1 vCPU) that scale to zero when not in use.

Article originally posted on InfoQ. Visit InfoQ

Many large language models (LLMs) have become available recently, both closed and open source, further leading to the creation of combined models known as Multimodal LLMs (MLLMs). Yet, few or none of them unveil what design choices were made to create them, say Apple researchers who distilled principles and lessons to design state-of-the-art (SOTA) Multimodal LLMs.

Multimodal large language models are built by combining a large language model and a vision foundation model into a single model. MMLMs, which according to Apple researchers “are emerging as the next frontier in foundation models”, aim at consuming image and text inputs to generate text data in a way that outperforms the foundation models they build upon.

Apple researchers focused on two aspects of the process that lead to the creation of MLLMs: decisions about the model architecture and choices for pre-training data.

On the first front, they found that image resolution, visual encoder loss and capacity, and visual encoder pre-training data are the three most important design aspects. On the contrary, architectural decisions regarding how visual data is fed into the LLM do not seem to affect the resulting model performance.

Regarding pre-training, the researchers analyzed three different approaches —image-caption, interleaved image-text, and text-only data— in few-shot, zero-shot, and text-only contexts. Zero-shot models are trained to recognize and classify objects or concepts without necessarily having previously seen any examples of them. In few-shot training, the focus is instead on models that can make accurate predictions based on training that includes only a very small number of labeled examples.

The outcome was that interleaved and text-only training data is key for few-shot and text-only model performance, while caption data is key for zero-shot models.

To prove their results, the researchers built a family of models, dubbed MM1, outperforming current state-of-the-art models, including Emu2, Flamingo, and IDEFICS. Benchmarking was done on captioning, where the model provides a descriptive caption of an image, and visual question answering, where the model answers questions about an image and helps understand its content.

Thanks to large-scale multimodal pre-training […] MM1 enjoys appealing properties such as in-context predictions, multi-image and chain-of-thought reasoning. MM1 also enables strong few-shot learning capability after instruction tuning. These strong results demonstrate that the presented recipe for building MLLMs translates the design principles to a competitive model at scale.

As the researchers explain in their paper, to get these levels of performance with MM1, they investigated different image encoders as well as ways of connecting them to LLMs; different types of data and how to set weights; and how to train the MLLM, including its hyperparameters. Their results include insights such as the importance of image resolution, model size, training data composition, and so on, which they hope can provide a solid foundation for the community to build stronger models across multiple architectures and data strategies.

Article originally posted on InfoQ. Visit InfoQ

Transcript

Hartshorne: I’m here to talk about a time in Honeycomb’s life. As a company, we had been around for several years and the business was slowly growing. We were getting more customers. As the traffic grew, some scaling issues come up, and they are managed. This is great. We could keep growing and a different thing comes up, and it’s managed. As the pace of growth goes along, these things happen. Each time, there are side effects that push out a little bit further. What you wind up with is a system that is still there and working well, and just creaking. It starts to push its boundaries more frequently in different ways. It’s not patterns, it’s rough. We saw as a business, we wanted to grow. We’re realizing that what got us to that part in our company’s life would not get us to the next one. I would like to talk a little bit about how we managed that.

Background

My name is Ben Hartshorne. I’m a Principal Engineer at Honeycomb. I worked on this talk with Jess Mink. They joined Honeycomb as a senior director of platform engineering when we were facing this. They started, and we said, “We would like to double the size of our business, please help.” We needed to get a lot of work that was on the periphery here, somehow scheduled in the face of growing the business and increasing the product, and everything else. The thesis is relatively straightforward. A compelling business case is what will get your work scheduled. We need to understand, what does a compelling business case look like? How do we get there? How do we ensure that we can do this for technical work? Because the company has been growing and the product organization has been growing, and there’s more sophistication around deciding what we should build. We need to match that.

Understanding Priorities

I want to start by thinking about what does it mean to understand priorities, just a little bit. It’s a truism in the startup world and probably in all of the business world, that there’s always more work than you can possibly do. There’s this herd of features out there, they are beautiful. They are product ideas. They’re things you can do, new things you can build, ways you can improve. They all must go through this gate of your engineering capacity. You can only do so much work. We’re starting from this perspective of there’s no way we can do everything. The engineer’s view on this herd is on the inside of this fence. There’s this line of beautiful work coming in at you, it is full. You have your roadmap. You have all of your work queued up. You have the new products to build, the existing ones to improve. You look at this and you think, how can I possibly do the work that I see that we need to do in order to maintain this business? When do I schedule this? Can we close the gate for a little bit so we can sweep the entrance? That’s hard. Take for a moment the other perspective, the product managers, the ones who are creating this curated list of things for you to do. This is their view. They see this enormous pile of things they would like to build, ways the business can improve, and they’re not going anywhere. Why isn’t engineering doing our work? They ask. They see this as an enormous opportunity, and need to figure out, engineering has the capacity that they do. We need to understand which sheep to offer them, which are going to make it through the gate first. This is a little bit about what they do.

Let’s imagine this sheep in the front here. This sheep in the front will reduce your hosting costs by 20%. Very good sheep. We should do this. Any project that might show up has to be considered in the context of what the business is doing, what’s currently going on everywhere. Imagine we are a primary color company who does some web search, and we have the opportunity to reduce our hosting costs by 20%. This is a career making move. This is going to happen. It doesn’t matter really. It matters a little bit how much it costs. This is a project that is absolutely going to happen. Picture this middle group here, classic startup scene, early in their business, they have the same opportunity to reduce their hosting costs by 20%. Their hosting cost is what, like $300 a month? No, you’re not going to do that. Then there’s all of the states in between where your business might be in a merger, or it might be in a reorg, or there might be something going on. You have this obvious benefit, reduce hosting costs, but it’s really not clear whether you should do it. If the reorg isn’t a complete, and the entire product that’s being hosted there is going to be disbanded, no, don’t go spend time reducing the costs to host it. The same project, the same decision, the same idea may get a very different result depending upon what’s going on in the rest of the business.

What’s the Prioritization Process?

Let’s look a little bit more at how to choose which of these things to do. It’s not always quite so simple as an enormous cost reduction. Product management has come up in this track before. There are sophisticated tools to help product managers take this enormous mountain of ideas and input and form it into the things that they’re going to present for engineering, to understand whether or not one of these things should be done. This input comes in many forms. There’s feedback from customers. There’s ideas sprouted from anywhere, really. There’s appreciation for a specific aspect of your business. All of these come together. They come in separately, but they need to be understood to be representations of a thing that the business does, a service that it provides, a problem that it’s solving. You can look at each of these different bits, whether it’s a piece of customer feedback that comes in, and try and reframe it so that it is shaped like a problem that the business is fulfilling. This process is managed for every one of those sheep we saw. What we wind up with is this enormous corpus of data about all of the different problems that we are confronting as a business, which ones affect our customers in one way or the other. Some of them are blocking new customers. Some of them are allowing the expansion of existing customers. Each one has this depth to it that has been accumulated by the product organization.

Once you get an understanding of these ideas, you can try and tease them apart into a couple of categories. There’s the obvious good, the obvious bad, the big, squishy middle. I particularly love this graph. I first came across it in a workshop by a guy named Jeff Patton. When I went to look for the image that he used, found he credited it to a graph called Constable’s truth curve, which was in fact presented here at QCon in 2013. On the left there you have the size of a bet you’re willing to make for this particular piece of work. On the top, going across is how much you expect to get from that. Let me give you an example. I have a new feature that is going to make you several millions of dollars as a business. It’s going to take quite a lot of time. It’s up there with the green dollar sign. You ask me, “So you’re proposing this thing. We recognize all development is a bet. How confident are you? You say it’s going to bring in this much money? What’s the likelihood you’re wrong?” I say, “I think it is, and if I wrong, I’ll buy you a cup of coffee.” Maybe don’t take that bet. The green is equally clear. You have some relatively easy task that has a very clear return. You’re very confident, there’s no question about whether this would benefit the business. I’m willing to give you my car if I’m wrong. Yes, we should do that. That’s in the nice green section up there. The squishy middle is like most of everything else. We need tools to help us explore these ideas and better understand where they fit. In hopes of pushing them out of that squishy green middle, into one of the other edges. There are a couple of good thought organizing frameworks out there. These is just two samples, an opportunity canvas, a learning canvas. They’re both ways of taking your understanding of the problem, of the people it will help, of the people it might not help, of the cost, structuring it in a way that facilitates some of this conversation.

Ultimately, the goal of this process is to clarify the ROI, the return on investment of any given piece of work. We need to find out who this work will influence, who it will impact, how much, and how much those users care about this particular thing. If it makes a big difference to some aspect that they never use, maybe not so important. A big part of the process of product management is understanding all of these questions. I’ve spent a while talking about this, it seems like a lot of work. We’re here to talk about tech debt. The thing is, yes, it is expensive, and software is more expensive. This has come up a couple of times in this track too. Engineers are expensive. Engineering efforts are expensive. I don’t want to just focus on us as individuals doing this work. You’ll hear somebody say, this is about one engineers’ worth of salary for a year or two for this project. If it’ll take us three months, we should do it, that sort of thing. That’s not enough. There’s a metric out there published by a number of public companies, it represents revenue per employee. We’re beyond engineering here, it’s not revenue per engineer, revenue per employee. This is what each employee of the company represents in terms of value that they’re building in order to bring the company to a good place. If we do a little quick math, engineering is usually maybe 30%, 35% of the company. Look at those numbers a little bit. We’re talking about a million dollars per engineer, in terms of the revenue they’re expected to generate through their work. For high performing companies, this is even higher, $2 million to $5 million.

Our time is expensive, it is worth a lot. We need to recognize that in order to value the work that the product organization has been putting into how they’re curating all of those sheep and to choose which ones they’re going to let through this gate. They are putting an enormous amount of respect on our teams, in order to choose the most impactful projects for us to build. When we think about tech debt, we think about a thing that we need to do that’s outside of this regular stream of work. We think that our piece of work should jump the queue. “This database upgrade is important. We have to do it.” Do we? Maybe not. What’s the business value? What’s the return? How can we measure this piece of work and this feature in the same language, the same framework as all of this effort that’s been put towards the rest of the work that is coming to us from the rest of the organization? Hang on to that question for a moment.

Recap (Understanding the Priorities)

We talked about, obviously, we can’t do all the work and the prioritization, we’re going to base strongly in a business case. Both the return we get for doing the work, also the cost of the investment we put into doing the work, and then that balance. The balance between those must exist in the context of the business and what’s important for the business at that time. The cost of the work, very good.

Selling the Impact or Your Project

Let’s talk about what we get back. What are we going to get for this project that we need to do? We’ve seen our work queues look like this. We need to upgrade the thing. We need to move the other bit. We need to refactor this. You all know what this is. This is tech debt. We can tell. We look at it, we see, yes. In order to focus it back on the business, so I want to think a little bit more about what tech debt is. It’s hard to say what tech debt is, so let’s answer the easier question instead. What do we do here? This is a wonderful graph, the VP of Engineering at Honeycomb put up in a blog post, understanding what goes into engineering work. On the left side, we have all the things that are customer facing, the features we actually build. Also, customer escalations and incident response, bug fixes. On the right side is all of the stuff that’s behind the scenes. Customers only see that part when something goes wrong: toolchains, refactors, training, testing, compliance, upgrades. Let’s split it the other way, up top. We think engineers are responsible for writing code, that’s true. The stuff up top, this is in code. This is adding new features, removing features. It’s also the upgrades. On the bottom, we have all of the attributes of running a system that are in production that are not in code. This is going to be part of your CI/CD system and your deploys, your training, your QA. Also, your incident response, the things you need to do that are part of owning your work in a modern software SaaS system. All of this is part of an engineer’s regular job.

Which one is tech debt? I think of tech debt normally in terms of refactoring of dependency upgrades, of work that you have to do in order to reform the product to fit some new business shape. Really, every category outlined in red here can be described as technical debt. I loved one of the examples from yesterday about doing a feature flag triage and leaving your feature in there just long enough so that you can reuse your feature flag when you’re putting in the next one. You delay the removing some code. It’s not just the stuff in the back. It’s not just the stuff that you think of in terms of refactoring. There’s a problem here, when we’re using a term so consistently, and we all have different definitions, it really muddies the conversation. When I’m talking to somebody, and I say, here’s this piece of tech debt, we need to spend some time on it, and they hear something that’s in a completely different portion of the circle. It does not aid the communication. Perhaps we can just stop using the term. We’re going to need to talk about work in a business context anyways. The goal of this workflow is the same as getting it on the roadmap.

Language is Important

As engineers, one of the things that we do is translate language that we see into understanding how that will affect us and our systems. There’s an observability company out there whose website has a big banner on the front, “Find your most perplexing application issues.” When I look at that, when we as engineers look at that, we’re like, how? I want to fit that into my product. I want to understand how I can rely on it, where I can rely on it. Where it will help me, where it won’t. How do you do this? What is it? We’ll make a translation in our head. Honeycomb is a tracing product. I believe it is a good example of what goes on in our heads. This translation is very useful when we’re building systems. When we’re trying to explain business value, it leads us to talk about, we need to upgrade the database. Not so helpful. We’re here to flip that around a little bit. As we are trying to understand how to prioritize and schedule the work that we see within engineering, we need to understand how to switch from talking about what it is that we need to do, upgrade the database, and start talking about why we need to do it so that we can connect it back to this business case. Sometimes that takes a couple of tries. We talked about needing to upgrade this database. Or maybe not. Do we, really? Let’s ask the team, why do we need to upgrade this database? They tell us, actually, it’s hitting its end of life. It’s an EoL database, we need to get off of it. Do we, really? It’s still serving our queries. In this mythical world, even that isn’t good enough. Let’s say that our hosting provider says, we will not run EoL software, so you have four months, and then we are shutting it off. Now we see, ok, what happens when they shut off our database? The entire product stops working. We need to upgrade that database, because we want the product to continue. If you go to your sales reps and say, would you like our product to stop working next September? They will say, no, please schedule the work you need to do. I would like you to fix that, how can I help? Who should I talk to? I will help you get this work scheduled, because it is clearly connected to ensuring that the business can continue. Hosting providers usually don’t go in there and flip off your database just because you’re running an EoL version of MySQL.

I want to tell another story. This one’s a little different, because I didn’t actually get the business case until after it had happened. We were running an old version of a LaunchDarkly library. LaunchDarkly does feature flagging. It was fine. It worked. There was one team that they had some extra time, they upgraded it, they had a thing they wanted to try. There was a shift in the model of this library from thinking about a user as the thing that manages a flag, to thinking about a context. Intentionally more broad. The context pulls in all sorts of other attributes of where this flag is being executed. This team used that to flag on their feature for a single node and get a very nice comparison graph. This was awesome. Within just a short period, many other teams were doing the same thing and getting to see, ok, I can do a percentage rollout, rollback. I can flip it on to these. We were used to only flipping it on and some of the data that is more relevant to our application, it opened up a whole new window. We saw, we can safely canary changes in a way that we couldn’t. We can do A/B testing in a way that we couldn’t. This is amazing. If we had been able to see that business case that we can make our engineering teams able to deploy their software more safely, more quickly, with more confidence around the change, yet better understanding of the impact of the change, we would have been able to prioritize that work. It was lucky that it was not so much effort and that one team had the space to make it happen. This idea of finding the business case in order to do the work, sometimes it takes some effort to understand what is going to happen with this. That can feel a little squishy.

Back It with Data

We are engineering teams here. We have a magical power. We can create data. We can find data. Our systems give us data. If they don’t have the data we need, we can make changes to those systems. We can commit code to get more data. We can use this to back up all of our arguments. When we get one of these conversations like, I think it’s going to do this, and someone else says I think it’s going to do that. We can find out. SLOs, top of the list there. They’re my favorite tool for mapping business value back to a technical metric, because in their formation, they are representing the experience of the user. They’re encoding, what does it mean for a given experience to be successful or not? How many of those experiences are we allowed to have fail? Over what period of time before we hit a threshold where our users and our customers get angry and leave?

I would love to spend the rest of the time talking about SLOs. Liz Fong-Jones did a talk, it’s published on InfoQ, on production excellence with a fantastic segment on SLOs. They’re not alone in our sources of data, we have all of our metrics showing various limits of our services. We get to do chaos experiments. When we have a service, and we’re not quite sure what that limit is, and we need to know more, we can artificially constrain that service in order to push its boundaries and find out what we would otherwise find out in an incident, six months down the road. Of course, there are those incidents. They really are two sides of the same coin. I don’t want to leave out qualitative surveys. Often a piece of technical engineering work that fits into this category of tech debt may feel like its target is not a specific amount of money for the business, but facilitating one’s experience in a certain part of the code base. There’s this idea of haunted graveyards, the section of the code where people fear to tread, because it’s dangerous, it’s complicated. Cleaning those up is valuable, to allow the business to move forward. Some qualitative surveys can really lend credence to this as you ask your engineers, how was it to build this new feature? How was your feature flag experience? With all of this data, it’s easy to fall back into those old habits. We need to upgrade the database because the CPU is too high. Who cares? You got to bring it back in business context, take that language about the CPU is too high and translate it to, what effect will that have on our ability to run our business? We’re back to our list here. We want to get this work prioritized. To do so, we understand what is important to the business here, now. We take the projects we need to do that we see are obvious and understand how to translate that into something that’s based on impact, rather than the technical change that we wish to make. We find the data we need to back it up.

I want to take you back to the Honeycomb experience of a year and a half ago, with this system that has been growing, and it’s starting to creak and fail in other interesting ways. Let’s run this playbook. Let’s see what happens. Start with understand the priorities. Honeycomb is a SaaS business. We are software as a service. ARR rules everything, this is annual recurring revenue. This is how much people are paying us, whether it’s monthly, or yearly. It includes people that decide to leave, and end their contract with Honeycomb. It is a big deal. ARR is this number that we have on our fundraising decks that we talk to the board about, that we make projections for. This is what we are organizing our business around, saying we’re going to get to this amount of revenue and we’re going to do it in this amount of time. I know there are many talks here about reducing costs and expanding revenue. For many SaaS companies earlier in their life, the balance is heavily skewed towards revenue. ARR is the thing, so let’s use it.

We get a dollar number of our ARR target, but that doesn’t translate to upgrading a database. By talking to the sales and product team, we get more detail, because this number is such a big deal, there’s a lot of research and work that goes into building it. It’s not just a number. It’s not just throwing a dart or watching a curve and saying, we’re going to make it there. No, they know we’re going to get this many customers of this type and we’re going to see these many upgrades and we’re going to see this many folks leave. All of this comes together to form that top line number. It’s wonderful that we have descriptions of who we’re going to get, still doesn’t help because I don’t know how to work with dollars but they have more answers. When I ask, ok, you’re getting 10 more enterprise customers, what does that mean? Which of our current customers can I model this after? How are they going to use our product? They came back with answers. They’re like, ok, this group of customers, they’re going to be focused more on ingestion and they’re going to use fewer of these resources. These other ones over here, they’re going to be more of a balance. Those ones there are going to be very heavy on the API. Now we’re getting onto my turf. We’re getting to numbers that I can see, the numbers that I have grasp about. I can now use this information to connect it back to the business value that’s going to come from each of those things. We need to take this data and map it to our infrastructure. Honeycomb is a macroservice architecture. We have a smallish number of relatively large services, they’re all named after dogs. We’re applying these to our infrastructure here.

How Does This Service Scale?

For each of these teams that have a dog under their care, we ask these teams, please spend some time understanding your service, writing about your service, and reflecting upon how it fits in here. Start with just an overview of the service. Then think about where the bottlenecks are, how the service scales. Then look at the sales targets that we covered that we have these numbers for, and write down which ones of those are applicable to your service. Maybe ingestion matters, maybe it doesn’t, depending on what the service consumes, who it talks to. Some are important, some aren’t. This is where we involved all of engineering. Every team did this. It didn’t take all that long. Our SRE team is already embedded in each of them, and they helped by guiding them through some of these processes to write down and give a unified format to these reports. As an example, our primary API service, this is the front door of Honeycomb’s ingestion. It accepts all of the telemetry from our customers. It does some stuff to it, authentication, authorization, validation, transformation. It sends it along to the downstream service, which is a Kafka cluster, for transmission to our storage engine. It has some dependencies. It depends upon a database for authentication tokens. It depends upon a Redis cluster for some rate limiting and things. The upstream services are our customers. The downstream services, it’s Kafka. The sales target affecting the service is clear. It’s the telemetry that our customers are sending us. When thinking about how it scales, it is a standard horizontally scaling service, stateless. These are the canonical scaling things. It can grow as wide as it wants, almost. Through some past experience, both planned and unplanned, we’ve discovered that as the number of nodes grows, the load on the database grows. When we get to about 100, the database starts to slow down, which means you need more nodes, which makes the database slow down, and it goes boom. That is clearly a scaling limit. We have it written down.

Now comes the fun question. We found the scaling limit. That’s great. We have to fix it, obviously. Maybe? Let’s take these sales targets and say, where are we going to be on this curve of our sales target? When are we going to hit this scaling limit? Is it within the next year? Do we need to fix it now? Is it a problem for future us? Let’s decide. For this one, as it turns out, we expect it to hit that volume by mid-year. The database thing was one that we had to focus on. We got this whole report, it did highlight the database load that we will hit before reaching our sales targets, also called out a couple other ones, and some that we don’t really know about. Those are a cue for running some chaos experiments, or other ways of testing our limits.

The Business Case

The business case came down to this one. It was very clear. We could go back to our sales team and say, we need to schedule some work to mitigate the load on this database as the service scales out. If we don’t, we will not hit the sales targets that you gave us. The answer is very easy. Yes, please schedule that work. How do we need to do it? We did this for all of our services and came up with a lovely Asana board showing some things to fix, some things to care about now. Some things that are probably relevant in the year. Some that aren’t clearly like yes or no, but might extend the runway, so we can keep them in our back pocket and understand that if we see something creeping up, that’s a thing we can do to make it work. There’s one tag, it’s called explody. You’ll see the label explody on some of these.

As an organization, there’s this fun side effect to this exercise that we got to talk about a couple of things that we don’t always cover in the regular day-to-day engineering. Explody is one of them. That’s a description of this API service horizontal scaling thing that I was talking about. It grows and you don’t really see much until you get to that threshold and then boom, everything comes crashing down. Those problems are scary and huge. Also, you don’t usually have a lot of time to react between once that cascade starts and when you hit the boom. It helps you understand a little bit about how to schedule that kind of work, the non-explody ones. As an example, if you have a service that runs regularly scheduled jobs, once per minute you run a job. Over time, that job takes longer, until it takes like 65 seconds. When it’s overlapping, the next one doesn’t start. That’s not explody. Some jobs won’t run, most will. It will degrade, but it will degrade slowly and it will degrade in a way that you can go back to those SLOs, and use that to help you figure out when to schedule it. Equally important problem, slightly different method of scheduling.

The last bit here, I don’t know how many of you have had a conversation with somebody when designing a system. We look at this diagram and this architecture and say, “No, that can’t scale. No, we can’t do it this way, that won’t scale.” It’s a struggle, because, yes, maybe it won’t, but do we care? Will the system work for us for now? There’s the classic adage of don’t design things the way Google does, because Google has 100 million times your traffic, and then you will never encounter those problems. If you follow that design, you’re doing yourself a disservice. These scaling problems, which ones to ignore, this mapping of sales targets to scaling limits is a very strong way to have that conversation and make it very clear that from the business perspective, we can wait for that one.

Close the Loop

We caught ourselves a strong business case, using the priorities of the business, time to fix some of these problems, and the choice to put some of the other ones off. We are now done. We have gotten this work scheduled, almost. There’s one more step here, which isn’t so much about getting the work scheduled in the first place but is about making our jobs as an engineering team, especially in a platform, a little bit easier the next time around. We need to celebrate. We need to show the success of this work that we’re doing. When we find one of these projects, and we finish it, we need to make it very clear that we put engineering effort towards a business problem in a way that has given us the capacity to grow the business in the way that we need to. Then, when we have finished it and it’s gone on, and we pass that threshold that was previously impossible, we can call it out again and say, remember this? Yes, we couldn’t have gotten here without it. This stuff is really important. It lends credence to our discussions in the future about what’s going to be happening when we have more technical work to schedule.

The last one, this was a fun experience from a near miss incident. There were some capacity issues, and we were struggling to grow the capacity of the service quickly enough. During the course of this near miss, one engineer bemoaned, it takes 3 whole minutes to spin up new capacity for this service. This engineer had been at Honeycomb for a couple of years, but that was still recent enough that they didn’t remember the time before we were in a Kubernetes cluster, when new capacity took 15 minutes to spin up. We put up a big project here. We’re going to change the way we’re doing our capacity and the way we run our services. We’re going to set a target of 10 minutes for new capacity, and new services to come up. No, let’s be aggressive, let’s set it to 5. Here we were recognizing it takes 3 whole minutes? This is great. This is amazing material because it gives us an opportunity to reflect on where we’ve come. For a lot of the platform work, for a lot of the tech debt work, that work is almost invisible. We need to find ways to bring it to light so that we can see as an organization the effect that this work is having on our longevity, on our speed, on our ability to deliver high quality software.

Summary

This is the recipe for getting technical debt work onto the roadmap. Understand what your business needs. Understand the impact of the projects you need to do to reduce that technical debt. Own the success of that business. Frame it in a way that is communicating that priority, not just the technical work. Use our power of data to back it up. Remember to ensure that you talk about the effect it had once you’re done, and that effect has been realized the business is exceeding those limits that it previously saw.

Questions and Answers

Participant 1: You didn’t bring one thing that was establishing a shared language or a shared vocabulary or framework for presenting the business benefits and things. What about scenarios where you have potential security issues? A scenario where I need to upgrade the library, because there is the potential for every bonus point on x. At which point now you’re just projecting what the potential damage should be. How do you solve that?

Hartshorne: How do we express security concerns in the language of business impact? I will admit, our security team is very good at that. I am not so good at it. There are a couple of ways. For larger businesses, there are frameworks that they’re required to operate under for security. There are agreements that one makes with both sub-processors and with customers around the response time to certain types of issues. You can use that to build an argument. If we are contracted to respond to a security incident within 30 days of being notified, we have to. You were asking about upgrading libraries because of potential flaws. That is definitely fuzzier. I was talking about how projecting projects that you’re going to make are bets. How much are you willing to bet that this is going to have that impact. Is it a cup of coffee? Is it a house? Framing security in terms of bets, it carries a little bit of a different feel to it. Yet, that is what’s going on. None of our services are 100% secure. We are doing risk analysis to understand which risks we are willing as a business to undertake and which we’re not. Do we have the recourse of the legal system if this particular risk manifests? Our business, and I think most businesses do have large documents describing their risk analysis for the current state of the world. The risks from inside, the risks from outside, the risks from third-party actors and so on. I think all of those are tools one can use to better assess the risk of a particular change. Those all come straight back again to business value.

Participant 2: I’ve often found that one of the hardest types of tech debt problems is to get to move on, to get buy-in. It’s the kind of problem where the benefit you’re going to get from solving it is freeing up your engineers to improve the velocity. You’re not solving tasks that really should be automated, but the amount of work to do and to automate it is going to take a couple weeks. We don’t do it, and in exchange for that sometimes we do it manually once every couple weeks, and it takes us few days to do it. It’s like an inducible cost from outside of the engineering organization, because you know that it’d free up a whole lot more engineering bandwidth to work on features and other things as a part of [inaudible 00:45:51]. I’m wondering if you have any thoughts on how to frame that as a business objective that would actually [inaudible 00:46:00].

Hartshorne: How do we schedule work that is toil that our engineers are doing successfully, because the job is getting done? There are a couple of ways. The first I want to highlight is the product philosophy of incremental improvements, shipping the smallest thing. There’s a wonderful diagram of how to build a car. One path starts with a set of wheels, and then a frame, and then some shell, and then the engine. At the end of it, you have a car, which, great, you can’t do anything until you get all the way to the end. The other says, maybe start with a skateboard, graduate to a bicycle, maybe a motorcycle, and finally, you’ll make your way to the car. I reference this, because a lot of those types of toil tasks can have small improvements that incrementally transform the task. As an example, you might start with a person that needs to roll the cluster by hand, and they go through and they kick each server in turn, and on they go. Simply writing that down in clear language, forming a checklist for it makes the next version faster. You can take two of those checkboxes and transform them into a script and it makes it faster. Each one of those is significantly easier to find the time to do than the entirety of the toil. That’s the first one.

The second is to bring up again, the idea of qualitative surveys. We ask our on-calls, every time they start a shift, and every time they finish a shift, a number of questions. It’s a light survey. Did you feel prepared for your shift? Do you think the level of alerting we have is too little too much, just about right? What came up during this shift? Did you feel stressed about any particular bit? What was the hardest part of this shift? Collecting those on a quarterly basis gives us some really good and unexpected insight into some of the types of toil that are coming up. Maybe it’s a flappy alert. Maybe it’s some service that needs to get kicked and need some improvements in its operational controls. Those are data. We can use those to say, our on-calls are spending 40% of their time chasing down false positives. This is wasted effort. That brings it back to expensive engineering time, and is, I think, a pretty good road into a compelling argument to spend some time improving the situation.

See more presentations with transcripts

Article originally posted on InfoQ. Visit InfoQ

Subscribe on:

Apple Podcasts
Google Podcasts
Soundcloud
Spotify
Overcast
Podcast Feed

.
From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.

Article originally posted on InfoQ. Visit InfoQ

HashiCorp recently released version 2.3 of Terraform Cloud Operator for Kubernetes with a new feature: the ability to initiate workspace runs declaratively. The Terraform Cloud Operator for Kubernetes was introduced in November 2023 to provide a Kubernetes-native experience while leveraging Terraform workflows.

The Terraform Cloud Operator allows users to manage Terraform Cloud resources with Kubernetes Custom Resource Definitions (CRD). This operator allows the users to provision infrastructure internal or external to the Kubernetes cluster directly from the Kubernetes control plane.

The Key Benefits of Terraform Cloud Operator are:

• Flexible resource management: Version 2 introduces multiple custom resources with dedicated controllers for various Terraform Cloud resources, enhancing flexibility and performance for managing custom resources concurrently in large-scale deployments.
• Namespace management: Tailor the operator’s watch scope to specific namespaces using the --namespace option, enabling fine-grained resource management tailored to your organizational needs.
• Configurable synchronization: Adjust the synchronization frequency between custom resources and Terraform Cloud with the --sync-period option, ensuring timely updates and operational smoothness.

In previous iterations of the Terraform Cloud Operator v2, initiating a run was limited to patching the restartedAt timestamp within the Module resource. This method proved non-intuitive, lacked universality across workspace types, and offered no control over run types, hampering migration efforts. However, version 2.3 addresses these challenges by enabling users to declaratively commence plan, apply, and refresh runs on workspaces, thereby empowering developers with enhanced self-service capabilities across all Operator-managed workspaces, including VCS-driven ones.

Version 2.3 introduces three new annotations within the Workspace custom resource to facilitate workspace run initiation:

• workspace.app.terraform.io/run-new: Set to true to trigger a new run.
• workspace.app.terraform.io/run-type: Control the run type (plan, apply, or refresh).
• workspace.app.terraform.io/run-terraform-version: Specify the Terraform version for speculative plan runs.

This is an example of Workspace Resource and Annotation:

apiVersion: app.terraform.io/v1alpha2
kind: Workspace
metadata:
  name: this
spec:
  organization: kubernetes-operator
  token:
    secretKeyRef:
      name: tfc-operator
      key: token
  name: kubernetes-operator

To immediately initiate a new apply run for the above workspace resource using kubectl:

kubectl annotate workspace this 
    workspace.app.terraform.io/run-new="true" 
    workspace.app.terraform.io/run-type=apply --overwrite

After successful execution, the annotation is reflected in the Workspace resource for observability:

apiVersion: app.terraform.io/v1alpha2
kind: Workspace
metadata:
  annotations:
    workspace.app.terraform.io/run-new: "true"
    workspace.app.terraform.io/run-type: apply
  name: this
spec:
  organization: kubernetes-operator
  token:
    secretKeyRef:
      name: tfc-operator
      key: token
  name: kubernetes-operator

Post-run completion, the operator automatically resets the run-new value to false.

With Terraform Cloud Operator v2.3, initiating and managing workspace runs becomes more intuitive, empowering teams to efficiently manage infrastructure while embracing Kubernetes-native experiences.

Posted on mongodb google news. Visit mongodb google news

MongoDB (NASDAQ:MDB – Get Free Report) had its price objective hoisted by analysts at Tigress Financial from $495.00 to $500.00 in a research note issued on Thursday, Benzinga reports. The firm currently has a “buy” rating on the stock. Tigress Financial’s price target suggests a potential upside of 38.45% from the stock’s current price.

Several other equities research analysts also recently issued reports on MDB. UBS Group restated a “neutral” rating and set a $410.00 price objective (down from $475.00) on shares of MongoDB in a research report on Thursday, January 4th. KeyCorp boosted their price objective on MongoDB from $500.00 to $543.00 and gave the company an “overweight” rating in a report on Wednesday, February 14th. Mizuho raised their target price on MongoDB from $330.00 to $420.00 and gave the company a “neutral” rating in a report on Wednesday, December 6th. Royal Bank of Canada upped their price objective on shares of MongoDB from $445.00 to $475.00 and gave the stock an “outperform” rating in a research report on Wednesday, December 6th. Finally, Stifel Nicolaus reiterated a “buy” rating and set a $435.00 price target on shares of MongoDB in a report on Thursday, March 14th. Two equities research analysts have rated the stock with a sell rating, three have given a hold rating and nineteen have assigned a buy rating to the company. Based on data from MarketBeat.com, the stock currently has a consensus rating of “Moderate Buy” and a consensus target price of $449.85.

View Our Latest Stock Analysis on MDB

MongoDB Stock Up 0.6 %

Shares of NASDAQ:MDB traded up $2.33 during midday trading on Thursday, hitting $361.13. The stock had a trading volume of 321,265 shares, compared to its average volume of 1,411,965. The company has a market cap of $26.07 billion, a P/E ratio of -145.82 and a beta of 1.24. The company has a quick ratio of 4.40, a current ratio of 4.40 and a debt-to-equity ratio of 1.07. The business’s 50-day simple moving average is $416.10 and its 200-day simple moving average is $390.35. MongoDB has a 12 month low of $198.72 and a 12 month high of $509.62.

Insider Transactions at MongoDB

In other MongoDB news, CFO Michael Lawrence Gordon sold 10,000 shares of the stock in a transaction on Thursday, February 8th. The stock was sold at an average price of $469.84, for a total value of $4,698,400.00. Following the completion of the transaction, the chief financial officer now owns 70,985 shares in the company, valued at $33,351,592.40. The transaction was disclosed in a filing with the SEC, which is available through the SEC website. In related news, CFO Michael Lawrence Gordon sold 10,000 shares of the company’s stock in a transaction that occurred on Thursday, February 8th. The stock was sold at an average price of $469.84, for a total value of $4,698,400.00. Following the sale, the chief financial officer now owns 70,985 shares in the company, valued at approximately $33,351,592.40. The transaction was disclosed in a document filed with the Securities & Exchange Commission, which is accessible through this link. Also, CEO Dev Ittycheria sold 33,000 shares of MongoDB stock in a transaction on Thursday, February 1st. The shares were sold at an average price of $405.77, for a total value of $13,390,410.00. Following the transaction, the chief executive officer now owns 198,166 shares of the company’s stock, valued at approximately $80,409,817.82. The disclosure for this sale can be found here. Over the last ninety days, insiders sold 54,607 shares of company stock worth $23,116,062. Company insiders own 4.80% of the company’s stock.

Institutional Inflows and Outflows

Hedge funds and other institutional investors have recently made changes to their positions in the company. Jennison Associates LLC boosted its stake in MongoDB by 87.8% during the 3rd quarter. Jennison Associates LLC now owns 3,733,964 shares of the company’s stock valued at $1,291,429,000 after purchasing an additional 1,745,231 shares during the last quarter. 1832 Asset Management L.P. boosted its position in shares of MongoDB by 3,283,771.0% during the fourth quarter. 1832 Asset Management L.P. now owns 1,018,000 shares of the company’s stock valued at $200,383,000 after buying an additional 1,017,969 shares during the last quarter. Norges Bank bought a new stake in MongoDB in the 4th quarter worth about $326,237,000. Axiom Investors LLC DE bought a new position in MongoDB during the 4th quarter valued at about $153,990,000. Finally, T. Rowe Price Investment Management Inc. boosted its holdings in shares of MongoDB by 77.4% during the 4th quarter. T. Rowe Price Investment Management Inc. now owns 568,803 shares of the company’s stock valued at $111,964,000 after acquiring an additional 248,133 shares during the last quarter. Institutional investors own 89.29% of the company’s stock.

About MongoDB

(Get Free Report)

MongoDB, Inc provides general purpose database platform worldwide. The company offers MongoDB Atlas, a hosted multi-cloud database-as-a-service solution; MongoDB Enterprise Advanced, a commercial database server for enterprise customers to run in the cloud, on-premise, or in a hybrid environment; and Community Server, a free-to-download version of its database, which includes the functionality that developers need to get started with MongoDB.

See Also

This instant news alert was generated by narrative science technology and financial data from MarketBeat in order to provide readers with the fastest and most accurate reporting. This story was reviewed by MarketBeat’s editorial team prior to publication. Please send any questions or comments about this story to contact@marketbeat.com.

Before you consider MongoDB, you’ll want to hear this.

MarketBeat keeps track of Wall Street’s top-rated and best performing research analysts and the stocks they recommend to their clients on a daily basis. MarketBeat has identified the five stocks that top analysts are quietly whispering to their clients to buy now before the broader market catches on… and MongoDB wasn’t on the list.

While MongoDB currently has a “Moderate Buy” rating among analysts, top-rated analysts believe these five stocks are better buys.

View The Five Stocks Here

Thinking about investing in Meta, Roblox, or Unity? Click the link to learn what streetwise investors need to know about the metaverse and public markets before making an investment.

Get This Free Report

Article originally posted on mongodb google news. Visit mongodb google news

Article originally posted on InfoQ. Visit InfoQ

According to Marijn Huizenveld, discipline is key to preventing accumulating technical debt. In order to be disciplined you should make it difficult to ignore the debt. Heuristics like fixing small issues immediately, agreeing on a timebox for improvement, and making messy things look messy, can help tame technical debt.

Marijn Huizendveld spoke about taming technical debt at OOP 2023 Digital.

To prevent the accumulation of technical debt and foster discipline, Huizendveld suggested using a highly visible area to show the debt inventory:

In the past, I’ve made enemies in the short term, both within and outside of the engineering community of organisations, by featuring the technical debt board prominently in the hallway. It was positioned in such a way that you could not get around it, literally. That is confrontational, because it shows where the organisation is failing, but all you do is make a problem apparent that is already there.

It takes some time to adjust for various roles that feel a responsibility towards the underlying causes, Huizendveld said. In the long term, high visibility results in frequent conversations about the tech debt inventory and what you are doing about it as a software organisation, he added.

Good technical debt is intentional, enables benefits for the organisation, and is controlled. Teams can use a disciplined approach for managing and repaying technical debt, Huizendveld said.

Not having any technical debt means liquidity, and liquidity means optionality, Huizendveld mentioned. People have a bias towards preventing losses, called loss aversion in bias literature. You can leverage this bias to provoke change.

Huizendveld provided some heuristics that have helped him tame technical debt:

• If you can fix it within five minutes, then you should.
• Try to address technical debt by improving your domain model. If that is too involved, you could resort to a technical hack. In the event that that is too involved, try to at least automate the solution. But sometimes even that is too difficult; in this case, make a checklist for the next time.
• Agree on a timebox for the improvement that you introduce with the team. How much time are you willing to invest in a small improvement? That defines your timebox. Now it is up to you and the team to honour that timebox, and if you exceed it, make a checklist and move on.
• Don’t fix it yourself, if it can be fixed by machines.
• If it is messy because you have a lot of debt, then make it look messy. Please don’t make a tidy list of your technical debt. The visual should inspire change.
• Only people with skin in the game are allowed to pay off debt, in order to prevent solutions that don’t work in practice.

Huizendveld suggested leaving alone areas that are stable. You don’t need to improve something that does not get changed, he argued.

InfoQ interviewed Marijn Huizendveld about dealing with technical debt.

InfoQ: How much technical debt do we typically see in software development?

Marijn Huizendveld: There was research done by Stripe Inc. in September 2018, described in the research paper The Developer Coefficient, which stated that “given an average workweek of 41,1 hours, developers spend circa 13,5 on technical debt, and including bad code it’s 17,3 hours.”

If you consider that the remaining time of 23,8 hours has to cover all the meetings and ceremonies that relate to planning work to be done, then you could infer that there is really not a lot of time left for actual work. Even if the numbers reported by developers on the impact of technical debt are exaggerated by 50%, it’s still a sizable amount.

InfoQ: How can we measure success in dealing with technical debt?

Huizendveld: There are a bunch of indicators:

• Team happiness should improve: track how people feel with a simple smiley rating, and use the answers to figure out if you’re on the right track. If people are frustrated because of the toil due to technical debt, then you know what to do. If people are happy because of the reduction of this toil, then you know doing more of it will bring even greater happiness.
• Unplanned work should go down: there is a reason why a shortcut is called a shortcut: it is not a thorough solution. In some cases, unplanned work is the consequence of shortcuts having a meeting with reality. This hinders the throughput of the team, because planned work has to be halted, to deal with the issues that are occurring in the production reality.
• Mean time to bug resolution should drop: there is a large category of technical debt that could be classified as technical debt in work processes. It’s the debt that hinders making changes to the system in the quickest way possible. For example, manual, error-prone steps to get the software into the hands of the users. When this category of technical debt is reduced, fixes are in production orders of magnitude quicker. On top of that, there is the fact that a lack of technical debt in the business logic makes it easier to address bugs, because the system behaves more predictably.
• And finally, your time to market should shorten: In the end, it is the cumulative effect of all of these small improvements that make the big improvement reality. Faster time to market is the compound effect of happy teams that can focus on the planned work without distractions, in a system that works for them, not against them.

Article originally posted on InfoQ. Visit InfoQ

Elon Musk announced that xAI would make its AI chatbot Grok open source, and now the release is accessible on GitHub and Hugging Face. This move enables researchers and developers to expand upon the model, influencing how xAI evolves Grok in the face of competition from tech giants like OpenAI, Meta, Google, Microsoft, and others. This milestone marks a significant turn in the field of AI, allowing other developers and experts in the field to access Grok’s code and related data for analysis and development.

The release of Grok as open source is a bold step that will open up new opportunities in AI research and development. Previously, industry-leading models like Mistral AI’s Mixtral and Meta’s Llama 2 dominated the AI research landscape. However, Grok stands out for its colossal size, boasting an impressive set of 314 billion parameters, nearly four times larger than its closest competitor, Llama 2.

This massive size suggests promising possibilities in terms of model accuracy and interaction capability. Grok’s weights, which are essential for its operation, are available for download, enabling developers to experiment with its structure and behavior.

@Gradio shares an X Post on all things essential about xAI’s Grok-1 release:

Now that Grok1 is open-sourced, its time we learn more about the model. All things essential about XAI’s Grok-1 release: 314B params – 8x33B MoE – 25% weights active on a token Base Model (Little) Better than Llama2 & GPT3.5 Apache2 Built in JAX & RUST 8-bit weights.

Elon Musk’s decision to take an open-source approach with Grok responds to the growing demand for transparency and collaboration in the field of AI. By sharing the code and data, Musk not only fosters innovation but also promotes accountability and public evaluation of the model.

Seeking an alternative to OpenAI and Google, Musk launched xAI with the aim of developing what he described as an AI focused on maximizing truth-seeking capabilities.

Open-source software, like Grok, offers a range of benefits for both developers and the community at large. Firstly, it allows for greater transparency and auditing, contributing to the trust and reliability of the software. Additionally, it fosters collaboration and knowledge sharing among developers worldwide, which can accelerate the pace of innovation.