jm + build   34

AWS CodeBuild Plugin - Jenkins - Jenkins Wiki
Trigger AWS CodeBuild jobs as build steps for a Jenkins project. :thinking_face_emoji:
jenkins  hacks  aws  codebuild  build  coding  ci 
5 days ago by jm
GTK+ switches build from Autotools to Meson
'The main change is that now GTK+ takes about ⅓ of the time to build
compared to the Autotools build, with likely bigger wins on older/less
powerful hardware; the Visual Studio support on Windows should be at
least a couple of orders of magnitude easier (shout out to Fan
Chun-wei for having spent so, so many hours ensuring that we could
even build on Windows with Visual Studio and MSVC); and maintaining
the build system should be equally easier for everyone on any platform
we currently support.'

Looking at http://mesonbuild.com/ it appears to be Python-based and
AL2-licensed open source.

On the downside, though, the Meson file is basically a Python script,
which is something I'm really not fond of :( more details at http://taint.org/2011/02/18/001527a.html .
meson  build  coding  dev  autotools  gtk+  python 
august 2017 by jm
Towards true continuous integration – Netflix TechBlog – Medium
Netflix discuss how they handle the eternal dependency-management problem which arises with lots of microservices:
Using the monorepo as our requirements specification, we began exploring alternative approaches to achieving the same benefits. What are the core problems that a monorepo approach strives to solve? Can we develop a solution that works within the confines of a traditional binary integration world, where code is shared? Our approach, while still experimental, can be distilled into three key features:

Publisher feedback — provide the owner of shared code fast feedback as to which of their consumers they just broke, both direct and transitive. Also, allow teams to block releases based on downstream breakages. Currently, our engineering culture puts sole responsibility on consumers to resolve these issues. By giving library owners feedback on the impact they have to the rest of Netflix, we expect them to take on additional responsibility.

Managed source — provide consumers with a means to safely increment library versions automatically as new versions are released. Since we are already testing each new library release against all downstreams, why not bump consumer versions and accelerate version adoption, safely.

Distributed refactoring — provide owners of shared code a means to quickly find and globally refactor consumers of their API. We have started by issuing pull requests en masse to all Git repositories containing a consumer of a particular Java API. We’ve run some early experiments and expect to invest more in this area going forward.


What I find interesting is that Amazon dealt effectively with the first two many years ago, in the form of their "Brazil" build system, and Google do the latter (with Refaster?). It would be amazing to see such a system released into an open source form, but maybe it's just too heavyweight for anyone other than a giant software company on the scale of a Google, Netflix or Amazon.
brazil  amazon  build  microservices  dependencies  coding  monorepo  netflix  google  refaster 
may 2017 by jm
Instead of containerization, give me strong config & deployment primitives
Reasonable list of things Docker does badly at the moment, and a call to fix them. I still think Docker/rkt are a solid approach, if not 100% there yet though
docker  containers  complaining  whinge  networking  swarm  deployment  architecture  build  packaging 
april 2017 by jm
grammarly/rocker
backward compatible replacement for Dockerfile. Yes, you can take any Dockerfile, rename it to Rockerfile and use rocker build instead of docker build. ... Rocker aims to solve the following use cases, which are painful with plain Docker:

Mount reusable volumes on build stage, so dependency management tools may use cache between builds.
Share ssh keys with build (for pulling private repos, etc.), while not leaving them in the resulting image.
Build and run application in different images, be able to easily pass an artifact from one image to another, ideally have this logic in a single Dockerfile.
Tag/Push images right from Dockerfiles.
Pass variables from shell build command so they can be substituted to a Dockerfile.
And more. These are the most critical issues that were blocking our adoption of Docker at Grammarly.

The most challenging part is caching. While implementing those features seems to be not a big deal, it's not trivial to do that just by utilising Docker’s image cache (the one that docker build does). Actually, it is the main reason why those features are still not in Docker. With Rocker we achieve this by introducing a set of trade-offs. Search this page for "trade-off" to find out more details.
docker  rocker  build  containers  dockerfiles 
may 2016 by jm
fiunchinho/dockerize-me
'Tired of copy/pasting Dockerfiles around? Not sure about best practices for Dockerfiles or Docker entry points? This tool lets you Dockerize your applications using best practices to define your Dockerfile and Docker entry point files.'

The best practices in question are defined here: https://github.com/docker-library/official-images#review-guidelines
docker  dockerfile  images  build  best-practices  alpine  containers 
may 2016 by jm
CD at LMAX: Testing into Production and Back Again
Chock-full of excellent build/test ideas from LMAX's Continuous Delivery setup. Lots of good ideas to steal
testing  lmax  build  test  continuous-delivery  dev 
may 2016 by jm
The Challenges of Container Configuration // Speaker Deck
Some good advice on Docker metadata/config from Gareth Rushgrove
docker  metadata  configuration  build  devops  dev  containers  slidfes 
may 2016 by jm
JitPack
Publish JVM and Android libraries direct from github -- it'll build and package a lib on the fly, caching them via CDN
build  github  java  maven  gradle  dependencies  packaging  libraries 
april 2016 by jm
Jenkins 2.0
built-in support for CI/CD deployment pipelines, driven from a checked-in DSL file. great stuff, very glad to see them going this direction. (via Eric)
via:eric  jenkins  ci  cd  deployment  pipelines  testing  automation  build 
march 2016 by jm
remind101/conveyor
'A fast build system for Docker images', open source, in Go, hooks into Github
build  ci  docker  github  go 
october 2015 by jm
Buck
A high-performance java build tool, from Facebook. Make-like
android  build  java  make  coding  facebook 
june 2015 by jm
How to change Gradle cache location
$GRADLE_USER_HOME, basically -- it may also be possible to set from the Gradle script itself too
gradle  build  caching  environment  unix  cache 
may 2015 by jm
[Nix-dev] Pulling a programs source code from a git repo
Nix supports building from git sha. excellent
nix  packaging  build  dev  ci 
march 2015 by jm
Stu Hood and Brian Degenhardt, Scala at Twitter, SF Scala @Twitter 20150217
'Stu Hood and Brian Degenhardt talk about the history of Scala at Twitter, from inception until today, covering 2.10 migration, the original Alex Payne’s presentation from way back, pants, and more. The first five years of Scala at Twitter and the years ahead!'

Very positive indeed on the monorepo concept.
monorepo  talks  scala  sfscala  stu-hood  twitter  pants  history  repos  build  projects  compilation  gradle  maven  sbt 
march 2015 by jm
Concourse
Concourse is a CI system composed of simple tools and ideas. It can express entire pipelines, integrating with arbitrary resources, or it can be used to execute one-off builds, either locally or in another CI system.
ci  concourse-ci  build  deployment  continuous-integration  continuous-deployment  devops 
march 2015 by jm
Try Server
Good terminology for this concept:
The try server runs a similar configuration to the continuous integration server, except that it is triggered not on commits but on "try job request", in order to test code pre-commit.

See also https://wiki.mozilla.org/ReleaseEngineering/TryServer for the Moz take on it.
build  ci  integration  try-server  jenkins  buildbot  chromium  development 
march 2015 by jm
On-Demand Jenkins Slaves With Amazon EC2
This is very likely where we'll be going for our acceptance tests in Swrve
testing  jenkins  ec2  spot-instances  scalability  auto-scaling  ops  build 
august 2014 by jm
How to take over the computer of any JVM developer
To prove how easy [MITM attacking Mavencentral JARs] is to do, I wrote dilettante, a man-in-the-middle proxy that intercepts JARs from maven central and injects malicious code into them. Proxying HTTP traffic through dilettante will backdoor any JARs downloaded from maven central. The backdoored version will retain their functionality, but display a nice message to the user when they use the library.
jars  dependencies  java  build  clojure  security  mitm  http  proxies  backdoors  scala  maven  gradle 
july 2014 by jm
Pillar
Manages migrations for your Cassandra data stores. Pillar grew from a desire to automatically manage Cassandra schema as code. Managing schema as code enables automated build and deployment, a foundational practice for an organization striving to achieve Continuous Delivery.

Pillar is to Cassandra what Rails ActiveRecord migrations or Play Evolutions are to relational databases with one key difference: Pillar is completely independent from any application development framework.
migrations  database  ops  pillar  cassandra  activerecord  scala  continuous-delivery  automation  build 
june 2014 by jm
Videos from the Continuous Delivery track at QCon SF 2012
Think we'll be watching some of these in work soon -- Jez Humble's talk (the last one) in particular looks good:

Amazon, Etsy, Google and Facebook are all primarily software development shops which command enormous amounts of resources. They are, to use Christopher Little’s metaphor, unicorns. How can the rest of us adopt continuous delivery? That’s the subject of my talk, which describes four case studies of organizations that adopted continuous delivery, with varying degrees of success.

One of my favourites – partly because it’s embedded software, not a website – is the story of HP’s LaserJet Firmware team, who re-architected their software around the principles of continuous delivery. People always want to know the business case for continuous delivery: the FutureSmart team provide one in the book they wrote that discusses how they did it.
continuous-integration  continuous-delivery  build  release  process  dev  deployment  videos  qcon  towatch  hp 
may 2013 by jm
Test-Driven Infrastructure with Chef
Interesting idea.
The book introduces “Infrastructure as Code,” test-driven development, Chef, and cucumber-chef, and then proceeds to a simple example using Chef to provision a shared Linux server. The recipes for the server are developed test-first, demonstrating both the technique and the workflow.
tdd  chef  server  provisioning  build  deploy  linux  coding  ops  sysadmin 
march 2013 by jm
Why djb redo won't be the Git of build systems
A counter-argument: "so, redo, from a conceptual point of view, has a really good and simple approach (very djb-y), and I'm sure it's an excellent tool for new projects, but for existing projects that already use make in a non-recursive fashion, it would a maintenance PITA. And that's why I conclude that redo in its current conceptual state will never be the Git of build systems. make is still more flexible, and even though it has its flaws, it's still good enough for most people, and also a de-facto standard."
redo  build  djb  building  make  compilation  from delicious
january 2011 by jm
The things make got right (and how to make it better)
jgc provides a good demonstration of how a general-purpose programming language tends to make a crappy DSL -- specifically Rakefiles
dsl  build  make  coding  jgc  languages  configuration  makefiles  rake  ruby  from delicious
january 2011 by jm
good Hacker News thread on djb's "redo"
YA make-replacement build system. the thread is better than the linked article, btw
hacker-news  via:fanf  make  build  djb  redo  compilation  building  coding  open-source  from delicious
january 2011 by jm

related tags

acquisitions  activerecord  alpine  amazon  android  apollo  architecture  auto-scaling  automation  autotools  aws  backdoors  bazel  best-practices  bintray  brazil  build  buildbot  building  cache  caching  cassandra  cd  chef  chromium  ci  clearcase  clojure  codebuild  coding  compilation  complaining  concourse-ci  configuration  containers  continuous-delivery  continuous-deployment  continuous-integration  database  dependencies  deploy  deployment  deterministic-builds  dev  development  devops  djb  docker  dockerfile  dockerfiles  dsl  ec2  environment  facebook  github  go  google  gradle  gtk+  hacker-news  hacks  history  hosting  hp  http  hudson  images  integration  ios  iphone  jars  java  jenkins  jgc  languages  libraries  linux  lmax  make  makefiles  maven  maven-central  meson  metadata  microservices  migrations  mitm  monorepo  netflix  networking  nix  nix-docker  nixos  nixpkgs  open-source  ops  oracle  oss  packages  packaging  pants  pillar  pipelines  process  projects  provisioning  proxies  python  qcon  rake  redo  refaster  release  repos  rocker  ruby  sbt  scala  scalability  scm  security  server  sfscala  slidfes  source-code  spot-instances  startups  stu-hood  sun  swarm  sysadmin  talks  tdd  test  testing  tips  towatch  try-server  twitter  unix  via:eric  via:fanf  via:jbaruch  videos  whinge 

Copy this bookmark:



description:


tags: