Category: Uncategorized

Article originally posted on InfoQ. Visit InfoQ

Heroku’s engineering team wrote about their journey from manual deployments to automated continuous deployments for Heroku Runtime, their managed environment for applications. They achieved this using Heroku primitives and a custom deployer tool.

The Heroku Runtime team builds and operates both single (Private Space/PS) and multi-tenant (Common Runtime/CR) environments. This includes container orchestration, routing and logging.  Previously, the team followed a deployment routine consisting of multiple manual steps – including signoff from the primary on-call engineer, allowing sufficient buffer for monitoring post-deployment, and monitoring multiple dashboards. This also had the overhead of waiting for things to stabilize before proceeding to other regions and services. It worked as the team was smaller then, and services and regions were limited to 2 prod environments in the US and the EU. With an increase in the number of team members and a long term project to re-architect the CR into multiple services, the team had to engage in an automation exercise and build an automated deployer tool.

InfoQ reached out to Bernerd Schaefer, Principal Member of the Technical Staff at Heroku, to learn more about the challenges they faced and the details of the solution.

The previous processes in place were dependent on team bandwidth and careful manual planning of the expected impact. Direwolf – a test platform – reported the status across regions. The growth of the team to 30+ members made this process cumbersome. Combined with the challenge of managing an architecture revamp for CR which would split their monolithic Ruby app for CR into multiple services, the team decided to push for complete automation. The app was running in 2 production environments and the manual steps led to higher coordination costs.

The team’s solution was to use existing Heroku primitives and a custom tool called cedar-service-deployer. Each service became part of a Pipeline, and sharded services were deployed across multiple staging and prod environments as part of a long-term project. The cedar-service-deployer tools – written in Go – scans pipelines for differences between stages. If it finds any, it runs checks to see if it can promote the code to the next stage. These checks include release age, sufficient time for integration tests, ongoing incidents, alerts that are firing, promoting only from the master branch, etc. Adding new checks requires a code change, says Schaefer, as the list is fixed. At the same time, he explains that teams can configure their own alerts:

Teams are able to configure which things are checked for individual services, particularly which alerts to monitor to determine the health of a service. For example, a service might have one alert checking that the service is up, one checking that its HTTP success rate is over 99%, and teams adding those services to the deployer would configure those alerts in a JSON file for the deployer service to monitor during releases.

Monitoring and alerting form an important part of the deployment, as they can indicate possible issues. Heroku uses Librato for collecting metrics and alerting. There are some other systems too for monitoring, but so far all of the services that are controlled by the deployer use Librato, says Schaefer.

Schaefer further elaborates on their philosophy of monitoring:

One of the things we’ve been pushing forward is baking monitoring into our standard service toolkit, so that every service has useful instrumentation by default. As services go into production and mature, they’ll probably need some custom instrumentation, of course. But the goal is that service developers can focus on and be experts in the features their service offers — without also needing to be experts in metrics collection, or tracing, or whatever else we want to know about how systems are operating.

Although in most cases the deployer can automatically decide whether to push or not, there is provision for manual override. Schaefer explains:

The system always allows operators to use the existing manual tooling to push out releases. We might do that during an incident to get a critical hotfix patch rolled out to the impacted environment. It’s rare to push out other changes during an incident, since we try to minimize the changes to production while one is open, and folks are rarely in such a rush to get things out, but that capability is there if it’s needed.

The stateless nature of the deployer means it works by trying to promote between any two stages in the pipeline, and is not tied to a single “release” version. This enables multiple commit points to be present concurrently at different stages of the deployment pipeline.

Article originally posted on InfoQ. Visit InfoQ

The TypeScript team announces the release of TypeScript 3.7, including optional chaining, nullish coalescing, assertion functions, and numerous other developer ergonomic improvements.

Optional chaining is a highly requested feature. As explained by TypeScript program manager Daniel Rosenwasser:

Optional chaining is issue #16 on our issue tracker. For context, there have been over 23,000 issues filed on the TypeScript issue tracker to date. This one was filed over five years ago – before there was even a formal proposal within TC39. For years, we’ve been asked to implement the feature, but our stance has long been not to conflict with potential ECMAScript proposals. Instead, our team recently took the steps to help drive the proposal to standardization, and ultimately to all JavaScript and TypeScript users. In fact, we became involved to the point where we were championing the proposal! With its advancement to stage 3, we’re comfortable and proud to release it as part of TypeScript 3.7.

Optional chaining allows developers to author code where some expressions can immediately stop running with a null or undefined condition through the new ?. operator for optional property access. For example, before optional chaining JavaScript or TypeScript code would look like this:
 

let x = (foo === null || foo === undefined) ? undefined : foo.bar();

if (foo && foo.bar && foo.bar.baz) { // ... }

With optional chaining, this gets replaced by:

let x = foo?.bar();

if (foo?.bar?.baz) { // ... }

The new ?. operator is slightly different than the && operation checks as the optional chaining operator treats valid data such as 0 or empty strings as truthy.

The same operator can also be used on arrays for optional element access if the array exists, and on function calls to call them if the function is not null or undefined.

return arr?.[0]; log?.(`Request started at ${new Date().toISOString()}`);

The TypeScript team also championed the new ECMAScript nullish coalescing operator, ?? . This new operator provides a falling back to a default value when dealing with null or undefined. The ?? operator replaces the use of || when providing an optional default value.

let x = foo ?? bar();

The TypeScript 3.7 release also introduces assertion functions that throw an error when something unexpected happens. Assertions in JavaScript often get used to guard against passing improper types. Before TypeScript 3.7, these checks could not get properly encoded, requiring either less strict checking or type assertions.

A TypeScript goal is to type existing JavaScript constructs in the least disruptive manner. One of TypeScript’s assertion function approaches is to ensure that the condition getting checked must be true throughout the containing scope. The other assertion signature does not check for a condition, instead of telling TypeScript that a specific variable or property has a different type.

This release is the most ambitious TypeScript release since the 3.0 release, with many additional features and improvements:

• Better support for never-Returning Functions
• Make --declaration and --allowJs work together
• More recursive type aliases to better support JSON and tuples
• useDefineForClassFields flag and declare property modifier to better align with the public class fields ECMAScript proposal
• Build-free editing with project references
• Uncalled function checks
• Flatter error reporting
• // @ts-nocheck in TypeScript files
• Semicolon formatter option
• Numerous website and TypeScript Playground improvements

TypeScript 3.7 also introduces a few potentially breaking changes to consider when upgrading to the latest version of TypeScript, mostly as a result of the new features:

• Changes in DOM type definitions to improve accuracy around nullness
• Class field mitigations
• Function truthy checks
• Local and imported type declarations now conflict
• API changes to enable the recursive type alias patterns, removal of the typeArguments property

The TypeScript community response has been overwhelmingly positive to this release, and most users report reasonably easy upgrades. For example, Matthew McEachen of PhotoStructure explains via Twitter:

Just updated @PhotoStructure (64k lines of TS) from v3.6.4 to v3.7.2, and only needed to tweak a couple types. Namely coercion from Promise<Maybe<Maybe<T>> to <Maybe<T> | PromiseMaybe<T>> used to work in 3.6, and needs some coaxing in 3.7. Thanks again for TS!`

For more information, read the official TypeScript 3.7 announcement.

The TypeScript team is already working on features for TypeScript 3.8, including new export * as ns syntax, top-level await, private fields, and more.

The TypeScript community recently concluded the second TSConf event on October 11th with TypeScript founder Anders Hejlsberg delivering the keynote on the TypeScript 3.7 release.

TypeScript is open source software available under the Apache 2 license. Contributions and feedback are encouraged via the TypeScript GitHub project and should follow the TypeScript contribution guidelines and Microsoft open source code of conduct.

Article originally posted on InfoQ. Visit InfoQ

The release of version 2.0 for the cloud native edge router Traefik introduces support for TCP routing and routing based on server name identification (SNI), request middleware, canary deployments and A/B testing, and a new dashboard and web UI. The latest release provides more tools for developers to configure and manage routes and improved cluster traffic visibility.

Developed by the cloud infrastructure software provider Containous, Traefik is an open source reverse proxy and load balancer written in Golang. Routing in version 1 of Traefik was limited to HTTP backends, but with the latest release, one of the earliest requested feature improvements has been addressed. Traefik 2.0 provides HTTP and TCP routing and can support both on the same port. Traefik EntryPoints define the port which will receive requests, whether HTTP or TCP. Routers connect incoming EntryPoint requests to the services that handle them. Over TLS, Traefik routes TCP requests based on the SNI.

Traefik 2.0 also introduces middleware for altering requests before or after routing them to their destination. The middleware functionality enables chaining middleware configuration into a reusable group, which can also be configured as a Kubernetes Custom Resource Definition (CRD). Traefik comes with several pre-defined middleware configurations, such as path manipulation, authentication mechanisms, circuit breaker, retry, error handling, and IP white listing.

Canary deployments and A/B testing can be achieved with service load balancers, which act as virtual services that forward requests to the actual services. To create a gradual deployment of a new service, the new service is first defined with a unique identifier. Then a service load balancer is created, which defines the proportion of traffic for each version of the service with the weight option.

http:
  services:
    canary-api:
      weighted:
        services:
        - name: my-api-v1
          weight: 3
        - name: my-api-v2
          weight: 1

Traefik can then be configured to route traffic to this service load balancer and the weights can be adjusted without having to redeploy the services themselves. Service load balancers can also be setup to duplicate traffic and send it to multiple services at the same time, which can be used to conduct A/B testing.

The new dashboard provides an overview of cluster traffic and the Traefik features that can be enabled. The dashboard helps users walk through each component of a system’s routing configuration.

Traefik Dashboard from the Containous blog

The Traefik documentation provides a guide for migrating from version 1 to version 2 as well as a migration tool that converts Ingress to Traefik IngressRoute resources, converts acme.json files from v1 to v2 format, and migrates the static configuration contained in the file traefik.toml to a Traefik v2 file. One of the key distinctions between the versions is that components such as frontends and backends have been replaced with the combination of routers, services, and middleware. Additional support for migrating can be found in the community forum.

Full details on Traefik version 2 can be found on the Containous blog.

Article originally posted on InfoQ. Visit InfoQ

BMW started their Agile and BizDevOps transformation in 2016. Ralf Waltram, head of R&D and Dr. Frank Ramsak, head of IT Governance, presented at DOES 2019 a state of their IT transformation journey to 100% agile, which represents according to them their largest transformation to date impacting their business model, as they are moving from being just car manufacturer to mobility providers. Waltram and Ramsak shared how they led their transformation, lessons and takeaways to help anyone who embarks on a new large agile and DevOps transformation.

BMW IT used to deliver large projects using waterfall methodology and had multiple applications with dependencies leading to releasing during weekends. IT was delivering using a bimodal IT model. 20 % of the development teams were leveraging agile and 80% were still waterfall. It became apparent that having 2 different ways of working and collaborating within IT meant for BMW having 2 different speeds and cultures. Teams on a 2-week sprint were delayed and impeded by the waterfall teams still working towards annual releases. The project structure naturally led to silos between development and operations. Work delivery also struggled from a silo between business and development. In order to address these 2 key challenges, IT moved from projects to products and to 100% BizDevOps to ensure a complete collaboration and transparency between IT and the business. They targeted 100% agile with the goal of being more flexible and developing a more customer-centric and value driven culture of execution, including a user experience capability.

BMW designed a holistic transformation approach focussed on 4 cornerstones: process, structure, technology and people and culture. Based on Waltram and Ramsak, the most important aspect of the change was to set up the people and the organizational culture to support lean and BizDevOps. They started with rolling out trainings and educating the organization about agile values and practices. They also structured the teams to bring the developers and operations together to form truly DevOps teams. They structured their IT portfolio around products and value streams allowing at portfolio level for minimum governance and maximum synchronization and autonomy. One of the most important change they made is how they fund their work. In the past, they funded projects and operations separately, and since 2019 they started funding products and teams to ensure that funding is tied to products and value. They also introduced new technologies to support microservice and cloud-based architectures to gradually replace their legacy monolithic applications. They developed an agile software development toolchain adopted today by 20,000 employees allowing them to streamline and synchronize their end to end development.

The business recognized the accomplishments and the value of the transformation. They saw an increase in release frequency that went from 12 per year to 2 per month and they saw a significant decrease in defects or in time to resolution.

Waltram and Ramsak recommended to have a bold transformation vision, but to start small with small increments and to scale up along the way. Working in small iterative increments allowed them to inspect and adjust the course very fast and learn along the way, which according to them is still a difficult thing to do in an organization driven by a culture of perfection and performance. They concluded by encouraging organizations to share and learn from other organizations and communities on how they transform their IT and business and scale their DevOps platform.

Article originally posted on InfoQ. Visit InfoQ

InfoQ Homepage Presentations Boost Your Team’s Productivity with a Powerful Visual Management!

• Related Editorial

Article originally posted on InfoQ. Visit InfoQ

After getting support for futures in version 1.36, Rust has finally stabilized async/.await in version 1.39. As Rust core team member Niko Matsakis explains, contrary to other languages, async/.await is a zero-cost abstraction in Rust.

Async/await support is implemented in Rust as syntactic sugar around futures, as it happens in most other languages:

An async function, which you can introduce by writing async fn instead of fn, does nothing other than to return a Future when called. This Future is a suspended computation which you can drive to completion by .awaiting it.

Rust is using a slightly different syntax though. This is how you can declare an async function and use it from another function:

async fn a_function() -> u32 { }

async fn another_function() {
    let r : u32 = a_function().await;
}

As you can see, Rust .await syntax differs somewhat from several other languages where await is implemented as a keyword, including TypeScript, C#, and many others. This choice makes it possible to combine more naturally awaiting an async function completion with the ? operator, which is used for seamless error propagation, such as in a_function().await?.

Most importantly, as Matsakis remarks, async/.await has no runtime cost in Rust. This is due to the fact that calling an async function does not schedule it for execution, as it happens in other languages. Instead, async functions are executed only when you call .await on their future return value. This makes them sort-of “lazy” and allows you to compose a sequence of futures without incurring any penalty.

Another advantage of async/.await is they integrate much better with Rust’s borrowing system, which is a big help since reasoning about borrows in async code is usually hard.

The Rust community reacted almost enthusiastically to the introduction of async/.await, which was regarded by many developers as the missing piece to make Rust prime-time-ready.

It was really hard to build asynchronous code until now. You had to clone objects used within futures. You had to chain asynchronous calls together. You had to bend over backwards to support conditional returns. Error messages weren’t very explanatory. You had limited access to documentation and tutorials to figure everything out. It was a process of walking over hot coals before becoming productive with asynchronous Rust.

Async/.await support has the potential to greatly improves Rust usability and developer productivity, say others, by simplifying greatly asynchronous programming and making it possible to write firmware/apps in allocation-free, single-threaded environments.

Some developers, though, consider async/.await insufficient and call for higher-level abstractions. One major limit that is highlighted is the need for all functions used in a async/.await call chain to be written with that paradigm in mind, otherwise your program will block at some point. This is a general, not Rust-specific issue, still its solution would require introducing a runtime system and breaking Rust zero-cost-abstraction philosophy.

An alternative design would have kept await, but supported async not as a function-modifier, but as an expression modifier. Unfortunately, as the async compiler transform is a static property of a function, this would break separate compilation.

As a final remark, the current implementation of async/.await in Rust is only considered a “minimal viable product” and further work will be done to improve and extend it. In particular, this should bring support for using async fn in trait definitions.

Article originally posted on InfoQ. Visit InfoQ

Can we truly be our authentic selves at work, or are we at times covering? Covering takes energy and can isolate people; companies that foster authenticity and remove barriers that inhibit people from being themselves tend to be more successful.

At Women in Tech Dublin 2019, a panel consisting of Mairead Cullen, CIO at Vodafone Ireland, and Ingrid Devin, director of Dell Women’s Entrepreneur Network, discussed being our authentic selves at work. The panel was led by Ruth Scott, radio, TV presenter, MC and event host.

Scott started the panel by asking what our authentic self is. Cullen stated that we are all unique in our skills, behavior, and who we are. She suggested to be comfortable with that, and share our authentic self in the workplace. “We should follow the path we made for ourselves, not follow a path defined by others,” she said.

Scott mentioned that at times people do cover themselves up, something that happens to everyone, as being yourself isn’t always easy. Devin confirmed this; being your authentic self can be hard, it can be a tough thing to do. She gave the example of people having to work in a culture that doesn’t mirror their values.

Being your authentic self isn’t binary; it’s not something you fully do or don’t, said Devin. She suggested to take small steps and be yourself in some aspects.

Cullen said that it can be subtle things that people are covering; small but potentially important things. As it takes energy to put up that mask, it would be better if people could be their authentic selves and take their masks off. It also prevents them from isolating themselves.

When people cannot “be themselves,” how does it affect companies? Devin stated that companies that allow people to behave authentically will be more successful.

Cullen mentioned that nobody is the same at work as they are at home. It is normal to be different in the workplace than at home, but when the individual feels this difference is forced upon them and consumes effort to maintain the difference, then it is a problem.

Companies have to think about what their brand is, said Cullen. This impacts the possibilities for people to associate themselves with the company and be themselves.

Devin mentioned that it can be small things that indicate people can be themselves, but they can mean an awful lot. It’s important to recognize and address things that inhibit people to be authentic at work.

Cullen stated that it’s important for big organizations to show their leadership from the top-down, but equally important is bottom-up leadership. Both are needed to make a difference, as well as sponsorship of diversity and inclusion (D&I). The senior leadership team needs to be a role model and actually put into practise the D&I policies. In addition, it is important to have the bottom-up input, with employees providing feedback and participating in employee resource groups.

Flexible working practices like working hours and unlimited holidays are really important to allow people to be themselves. They have to be lived in the organization to really work, Cullen said. She gave the example of a senior person in her company who took term time; the fact that it was a senior person doing this influenced others who were considering it.

When you hear someone say something inappropriate, it might be difficult to call it out. Most of us have been in such situations, said Scott. As you

Article originally posted on InfoQ. Visit InfoQ

InfoQ Homepage Presentations Sherlock Holmes Value Detection

• Related Editorial

• Related Vendor Content

Article originally posted on InfoQ. Visit InfoQ

Subscribe on:

Apple Podcasts Google Podcasts Soundcloud Spotify Overcast RSS Feed

About QCon

QCon is a practitioner-driven conference designed for technical team leads, architects, and project managers who influence software innovation in their teams. QCon takes place 8 times per year in London, New York, Munich, San Francisco, Sao Paolo, Beijing, Guangzhou & Shanghai. QCon London is at its 14th Edition and will take place Mar 2-5, 2020. 140+ expert practitioner speakers, 1600+ attendees and 18 tracks will cover topics driving the evolution of software development today. Visit qconlondon.com to get more details.

More about our podcasts

You can keep up-to-date with the podcasts via our RSS Feed, and they are available via SoundCloud, Apple Podcasts, Spotify, Overcast and the Google Podcast. From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.

Previous podcasts

Article originally posted on InfoQ. Visit InfoQ

Low-cost virtual reality (VR) platform Google Cardboard is now available as an open source project to let developers create new VR-powered apps and adapt existing ones to new devices. Google’s announcement comes a few weeks after the discontinuation of its Daydream VR platform.

With Cardboard and the Google VR software development kit (SDK), developers have created and distributed VR experiences across both Android and iOS devices, giving them the ability to reach millions of users.

Introduced in 2014, Google Cardboard is intended as a low-cost VR headset to encourage experimentation with VR applications. As its name implies, the viewer was originally made of cardboard and other low-cost components such as 45mm focal length lenses, a velcro fastener, and a few magnets, and relied on a smartphone to be used as a display.

More than 15 million Cardboard were sold worldwide, says Google, with over 160 million Cardboard-enabled app downloads. According to Google, the Cardboard contributed to the success of the YouTube Virtual Reality channel and made possible the creation of the education-focused Expeditions app. Google also released precise schematics and assembly instructions that enabled the creation of a number of variations of the original design which were available for as little as $5. Now, Google is trying to replicate this schema by open-sourcing the whole platform to inject new life in it.

While we’ve seen overall usage of Cardboard decline over time and we’re no longer actively developing the Google VR SDK, we still see consistent usage around entertainment and education experiences, like YouTube and Expeditions, and want to ensure that Cardboard’s no-frills, accessible-to-everyone approach to VR remains available.

Google Cardboard SDK is available on GitHub and includes support to create Android and iOS apps that use motion tracking and stereoscopic rendering to power immersive experiences.

The interest aroused by the Cardboard led Google to introduce a more advanced platform, Google Daydream VR, which was integrated within Android itself and required some hardware support found only in specific devices. As mentioned, Google discontinued its Daydream VR platform a few weeks ago due its declining adoption. Among the reasons Google gave to justify Daydream VR demise was people reluctance to part from their phones while using them as a VR viewer.

If you want to create VR experiences using Google Cardboard, head to the Cardboard developer portal, from where you can access the API reference and the get started guides.