mpm + testing   103

Co-evolving Tracing and Fault Injection with Box of Pain
Distributed systems are hard to reason about largely because of uncertainty about what may go wrong in a particular execution, and about whether the system will mitigate those faults. Tools that perturb executions can help test whether a system is robust to faults, while tools that observe executions can help better understand their system-wide effects. We present Box of Pain, a tracer and fault injector for unmodified distributed systems that addresses both concerns by interposing at the system call level and dynamically reconstructing the partial order of communication events based on causal relationships. Box of Pain’s lightweight approach to tracing and focus on simulating the effects of partial failures on communication rather than the failures themselves sets it apart from other tracing and fault injection systems. We present evidence of the promise of Box of Pain and its approach to lightweight observation and perturbation of distributed systems.
testing  observability 
4 weeks ago by mpm
Co-evolving Tracing and Fault Injection with Box of Pain
Distributed systems are hard to reason about largely because of uncertainty about what may go wrong in a particular execution, and about whether the system will mitigate those faults. Tools that perturb executions can help test whether a system is robust to faults, while tools that observe executions can help better understand their system-wide effects. We present Box of Pain, a tracer and fault injector for unmodified distributed systems that addresses both concerns by interposing at the system call level and dynamically reconstructing the partial order of communication events based on causal relationships. Box of Pain's lightweight approach to tracing and focus on simulating the effects of partial failures on communication rather than the failures themselves sets it apart from other tracing and fault injection systems. We present evidence of the promise of Box of Pain and its approach to lightweight observation and perturbation of distributed systems.
testing  monitoring 
8 weeks ago by mpm
iMarcoGovea/books
Books about Nodejs, Angular2, Agile, Clean Code, Docker, Golang, Microservices, REST, TDD, BDD, and Startups.
book  rest  testing  agile  go  design  ddd 
september 2018 by mpm
Testing Distributed Systems
List of resources on testing distributed systems
testing 
may 2018 by mpm
Verifying Strong Eventual Consistency in Distributed Systems
Data replication is used in distributed systems to maintain up-to-date copies of shared data across multiple computers in a network. However, despite decades of research, algorithms for achieving consistency in replicated systems are still poorly understood. Indeed, many published algorithms have later been shown to be incorrect, even some that were accompanied by supposed mechanised proofs of correctness. In this work, we focus on the correctness of Conflict-free Replicated Data Types (CRDTs), a class of algorithm that provides strong eventual consistency guarantees for replicated data. We develop a modular and reusable framework in the Isabelle/HOL interactive proof assistant for verifying the correctness of CRDT algorithms. We avoid correctness issues that have dogged previous mechanised proofs in this area by including a network model in our formalisation, and proving that our theorems hold in all possible network behaviours. Our axiomatic network model is a standard abstraction that accurately reflects the behaviour of real-world computer networks. Moreover, we identify an abstract convergence theorem, a property of order relations, which provides a formal definition of strong eventual consistency. We then obtain the first machine-checked correctness theorems for three concrete CRDTs: the Replicated Growable Array, the Observed-Remove Set, and an Increment-Decrement Counter. We find that our framework is highly reusable, developing proofs of correctness for the latter two CRDTs in a few hours and with relatively little CRDT-specific code.
consistency  crdt  testing 
august 2017 by mpm
Going down the rabbit hole with go-fuzz
Inspired by AFL, Dmitry Vyukov created go-fuzz, which is a fuzz testing tool for the Go programming language. It’s one of the most useful tools in the Go toolchain, but it’s not as widely known as it should be
testing  go 
july 2017 by mpm
premock
Mock C functions using the preprocessor
testing  c++ 
april 2017 by mpm
Solving the Water Jug Problem from Die Hard 3 with TLA+ and Hypothesis
Accessible example of the power of property based testing.
testing 
april 2017 by mpm
CharybdeFS
CharybdeFS is our new FUSE-based error injecting pass-through filesystem
testing 
march 2017 by mpm
CORDS
File-system fault injection framework for distributed storage systems
storage  testing  availability 
march 2017 by mpm
httplab
HTTPLabs let you inspect HTTP requests and forge responses
http  testing 
march 2017 by mpm
libbreakr
Catches use of non re-entrant glibc functions
concurrency  testing 
march 2017 by mpm
big-list-of-naughty-strings
The Big List of Naughty Strings is a list of strings which have a high probability of causing issues when used as user-input data.
testing 
january 2017 by mpm
tdd - How deep are your unit tests?
my philosophy is to test as little as possible to reach a given level of confidence (Kent Beck)
testing 
december 2016 by mpm
libfuzzer-workshop
Repository for materials of "Modern fuzzing of C/C++ Projects" workshop
c++  testing 
november 2016 by mpm
autocheck
Header-only C++11 library for QuickCheck (and later, SmallCheck) testing.
c++  testing 
september 2016 by mpm
Circumventing Fuzzing Roadblocks with Compiler Transformations
We build some LLVM passes which ‘deoptimize’ code generated by LLVM to increase code coverage with AFL (and potentially other feedback driven fuzzers, e.g. libFuzzer)
testing 
august 2016 by mpm
UdpPinger
UdpPinger is our high performance UDP packet generation, reflection and collection library
networking  performance  testing 
august 2016 by mpm
Encapsulation of asynchronous behaviour in distributed system of scripted autotests
This is an example how to create continuation-like monadic behaviour in C++11 (encapsulate async execution) which is used for specific example: writing asynchronous scripts for system of distributed autotests
c++  concurrency  testing 
july 2016 by mpm
RFC 5166 - Metrics for the Evaluation of Congestion Control Mechanisms
This document discusses the metrics to be considered in an evaluation of new or modified congestion control mechanisms for the Internet
networking  protocol  testing 
july 2016 by mpm
doctest
doctest - The lightest feature rich C++ single header testing framework for unit tests and TDD
c++  testing 
may 2016 by mpm
Diskspd Utility
A feature-rich and versatile storage testing tool, Diskspd (version 2.0.15) combines robust and granular IO workload definition with flexible runtime and output options, creating an ideal tool for synthetic storage subsystem testing and validation.
performance  testing  io 
april 2016 by mpm
FakeIt
C++ mocking made easy. A simple yet very expressive, headers only library for c++ mocking.
c++  testing 
april 2016 by mpm
trompeloeil
Header only C++14 mocking framework
c++  testing 
january 2016 by mpm
libfaketime
libfaketime modifies the system time for a single application
time  testing  linux 
january 2016 by mpm
scal
High-performance multicore-scalable data structures and benchmarks
c++  concurrency  performance  testing 
october 2015 by mpm
american fuzzy lop
American fuzzy lop is a security-oriented fuzzer that employs a novel type of compile-time instrumentation and genetic algorithms to automatically discover clean, interesting test cases that trigger new internal states in the targeted binary. This substantially improves the functional coverage for the fuzzed code.
testing 
october 2015 by mpm
lest
A modern, C++11-native, header-only, tiny framework for unit-tests, TDD and BDD
c++  testing 
september 2015 by mpm
rapidcheck
QuickCheck clone for C++ with the goal of being simple to use with as little boilerplate as possible
testing  c++ 
august 2015 by mpm
Dynamic Plain Paxos
The article describes a generic way to prove the correctness of paxos-based distributes systems during the change of configuration and uses it to prove the correctness of membership changes
paxos  testing 
july 2015 by mpm
jimfs
An in-memory file system for Java 7+
java  testing 
april 2015 by mpm
fulltrace
A complete ftrace- and uprobes-based tracer (user, libraries, kernel) for GNU/Linux
linux  performance  testing  managability 
march 2015 by mpm
Decrypting TLS Browser Traffic With Wireshark
It turns out that Firefox and Chrome both support logging the symmetric session key used to encrypt TLS traffic to a file.  You can then point Wireshark at said file and presto! decrypted TLS traffic
browser  confidentiality  testing  networking 
february 2015 by mpm
flint
An open-source lint program for C++
c++  testing 
december 2014 by mpm
Simple Testing Can Prevent Most Critical Failures: An Analysis of Production Failures in Distributed Data-Intensive Systems
Large, production quality distributed systems still fail periodically, and do so sometimes catastrophically, where most or all users experience an outage or data loss. We present the result of a comprehensive study investigating 198 randomly selected, user-reported failures that occurred on Cassandra, HBase, Hadoop Distributed File System (HDFS), Hadoop MapReduce, and Redis, with the goal of understanding how one or multiple faults eventually evolve into a user-visible failure. We found that from a testing point of view, almost all failures require only 3 or fewer nodes to reproduce, which is good news considering that these services typically run on a very large number of nodes. However, multiple inputs are needed to trigger the failures with the order between them being important. Finally, we found the error logs of these systems typically contain sufficient data on both the errors and the input events that triggered the failure, enabling the diagnose and the reproduction of the production failures.
database  testing  outage 
november 2014 by mpm
FIG: Library-Level Error Injection for Shared Libraries
FIG (originally, Fault Injection in Glibc) is a UNIX tool to generate and inject errors from shared libraries into applications. It can be used to simulate failures from the underlying system environment when run on glibc library routines. In this manner, it is able to test the ability of applications to handle faults from library routines. In addition, it includes a call logging tool, similar to strace, and an error injection interface that uses a control file to describe error sequences
reliability  testing 
september 2014 by mpm
Multiparty Session Actors
Actor coordination armoured with a suitable protocol description language has been a pressing problem in the actors community. We study the applicability of multiparty session type (MPST) protocols for verification of actor programs. We incorporate sessions to actors by introducing minimum additions to the model such as the notion of actor roles and protocol mailbox. The framework uses Scribble, which is a protocol description language based on multiparty session types. Our programming model supports actor-like syntax and runtime verification mechanism guaranteeing type-safety and progress of the communicating entities. An actor can implement multiple roles in a similar way as an object can implement multiple interfaces. Multiple roles allow for inter-concurrency in a single actor still preserving its progress property. We demonstrate our framework by designing and implementing a session actor library in Python and its runtime verification mechanism.
actors  testing 
june 2014 by mpm
Fenix Web Server
A simple static desktop web server. Because simple stuff shouldn't need Apache, IIS, or NGINX.
http  testing 
june 2014 by mpm
shellcheck
ShellCheck, a static analysis tool for shell scripts
bash  testing 
april 2014 by mpm
Murphi
an enumerative (explicit state) model checker, with its own input language (also called Murphi) which is a guard -> action notation similar to Unity, which are repeatedely executed in an infinite loop.
testing 
march 2014 by mpm
blockade
Blockade is a utility for testing network failures and partitions in distributed applications. Blockade uses Docker containers to run application processes and manages the network from the host system to create various failure scenarios
testing  networking  fault-tolerance 
february 2014 by mpm
partitions
Partitions.tcl is a small Tcl program to simulate network partitions among a set of real computers
networking  testing  fault-tolerance 
december 2013 by mpm
Catch
C++ AutomatedTest Cases in Headers
testing  c++ 
april 2013 by mpm
UnitTest++
UnitTest++ is a lightweight unit testing framework for C++. It was designed to do test-driven development on a wide variety of platforms. Simplicity, portability, speed, and small footprint are all very important aspects of UnitTest++.
testing  c++ 
april 2013 by mpm
freud
Freud gives the users an easy way to define their own static analysis tests
java  testing 
march 2013 by mpm
likwid
This is an effort to develop easy to use but yet powerful performance tools for the GNU Linux operating system
performance  linux  testing 
march 2013 by mpm
Reproducing Network Research
Using Mininet-HiFi to reproduce published networking experiments
networking  testing 
february 2013 by mpm
ostinato
Ostinato is an open-source, cross-platform network packet crafter/traffic generator and analyzer with a friendly GUI. Craft and send packets of several streams with different protocols at different rates. For the full feature list see below.

Ostinato aims to be "Wireshark in Reverse" and become complementary to Wireshark
networking  testing 
february 2013 by mpm
Vaurien, the Chaos TCP Proxy
Vaurien is basically a Chaos Monkey for your TCP connections. Vaurien acts as a proxy between your application and any backend
networking  testing  tcp 
january 2013 by mpm
Using NFQUEUE and libnetfilter_queue
NFQUEUE is an iptables and ip6tables target which delegate the decision on packets to a userspace software
networking  linux  testing 
january 2013 by mpm
Igloo
Igloo is a unit testing framework for C++ that aims to stay out of your way and let you focus on what you want to test, help you create readable tests, have a syntax that doesn’t make you repeat yourself
c++  testing 
july 2012 by mpm
tcprstat
tcprtime is a protocol-agnostic libpcap-based tool for measuring TCP requests' response time in a server
networking  performance  testing  linux 
july 2012 by mpm
httpie
HTTPie is a CLI HTTP utility built out of frustration with existing tools. The goal is to make CLI interaction with HTTP-based services as human-friendly as possible. HTTPie does so by providing an http command that allows for issuing arbitrary HTTP requests using a simple and natural syntax and displaying colorized responses:
http  testing 
july 2012 by mpm
Netzob
NETZOB is an opensource tool which supports the expert in its operations of reverse engineering, evaluation and simulation of communication protocols
networking  testing 
june 2012 by mpm
Using lightweight modeling to understand chord
Correctness of the Chord ring-maintenance protocol would mean that the protocol can eventually repair all disruptions in the ring structure, given ample time and no further disruptions while it is working. In other words, it is "eventual reachability." Under the same assumptions about failure behavior as made in the Chord papers, no published version of Chord is correct
dht  availability  testing 
june 2012 by mpm
pathod
A pathological HTTP/S daemon, useful for testing and torturing HTTP clients
http  testing 
april 2012 by mpm
Fay: Extensible Distributed Tracing from Kernels to Clusters
Fay is a flexible platform for the efficient collection, processing, and analysis of software execution traces. Fay provides dynamic tracing through use of runtime instrumentation and distributed aggregation within machines and across clusters
distributed  testing  monitoring 
december 2011 by mpm
Pip: Detecting the Unexpected in Distributed Systems
Pip finds structural and performance bugs in distributed systems by comparing actual system behavior to expected system behavior. Actual system behavior is based on program annotations, interposition, or sniffing -- or some combination of the above. Expected system behavior is described in an external specification of the system's call graph and communication protocols
distributed  testing  performance 
december 2011 by mpm
Using strace and lsof to track down process hangs
One common situation that I've run across as a sysadmin/devops guy is the dreaded "it's hanging and we don't know why" problem … If you follow the recipe in this post, you should be able to quickly identify and fix many cases that exhibit this behavior.
linux  testing  monitoring  managability 
october 2011 by mpm
ievms
Microsoft provides virtual machine disk images to facilitate website testing in multiple versions of IE, regardless of the host operating system. Unfortunately, setting these virtual machines up without Microsoft's VirtualPC can be extremely difficult. These scripts aim to facilitate that process using VirtualBox on Linux or OS X. With a single command, you can have IE6, IE7, IE8 and IE9 running in separate virtual machines
browser  testing 
september 2011 by mpm
Test Anything Protocol
The Test Anything Protocol (TAP) is a protocol to allow communication between unit tests and a test harness. It allows individual tests (TAP producers) to communicate test results to the testing harness in a language-agnostic way
testing  protocol  parser 
august 2011 by mpm
data-race-test
This project is about detecting data races (race conditions).
concurrency  testing  fault-prevention  c++ 
august 2011 by mpm
The Network Simulator - ns-2
Ns is a discrete event simulator targeted at networking research. Ns provides substantial support for simulation of TCP, routing, and multicast protocols over wired and wireless (local and satellite) networks
networking  testing 
june 2011 by mpm
The Java Network Simulator
JNS allows developers of networking protocols to simulate their protocols in a controlled environment. The simulator then produces a trace file (same format as NAM trace files) which can be viewed in a network animator such as Javis
networking  testing 
june 2011 by mpm
meck and eunit best practices
some useful tips for how to get the most out of [meck] when using eunit
erlang  testing 
june 2011 by mpm
PeerSim P2P Simulator
Peer-to-peer systems can be of a very large scale such as millions of nodes, which typically join and leave continously. These properties are very challenging to deal with. Evaluating a new protocol in a real environment, especially in its early stages of development, is not feasible.

PeerSim has been developed with extreme scalability and support for dynamicity in mind
p2p  testing 
june 2011 by mpm
Ns
Ns is a discrete event simulator targeted at networking research. Ns provides substantial support for simulation of TCP, routing, and multicast protocols over wired and wireless (local and satellite) networks
networking  testing 
may 2011 by mpm
Simulating EDGE and 3G Speeds
Using ipfw we can simulate slower connection speeds on our desktop
networking  testing 
april 2011 by mpm
msgdropsim
Simulator for message passing protocols, supporting (really) unfair process scheduling and dropped messages
protocol  testing  networking  erlang 
april 2011 by mpm
emetric
emetric creates a high level view of the resources that a running erlang system consumes over time. This is useful for long running stress tests
erlang  monitoring  testing 
march 2011 by mpm
httpstat.us
This is a super simple service for generating different HTTP codes.

It's useful for testing how your own scripts deal with varying responses
http  testing 
february 2011 by mpm
ifconfig.me
echo back http request information
testing  http 
december 2010 by mpm
Mininet
Mininet creates scalable software-defined networks on a single PC by using Linux processes in network namespaces
linux  networking  testing 
december 2010 by mpm
Add latency to localhost
To simulate a far away server, add RTT time to the localhost device
networking  testing 
december 2010 by mpm
Iperf
Iperf was developed by NLANR/DAST as a modern alternative for measuring maximum TCP and UDP bandwidth performance. Iperf allows the tuning of various parameters and UDP characteristics. Iperf reports bandwidth, delay jitter, datagram loss
networking  testing  monitoring 
november 2010 by mpm
Changing what time a process thinks it is with libfaketime
With libfaketime you can tell a process that the current time is something different from the machine's system clock
linux  testing 
november 2010 by mpm
Seven key ideas of real quality assurance
“’Quality’ is far more than bugs. Bugs are an interesting subject but there are 50-odd other qualities your stakeholders are interested in”.
testing 
november 2010 by mpm
Trouble-Maker
When installed and run, this project will randomly select a problem from its set of issues and make it happen on your system
linux  testing 
september 2010 by mpm
gremlins
Gremlins is a fault injector - it is an evil program that sits on a machine and does nasty things to other programs on that machine (or the machine overall).
testing  distributed 
august 2010 by mpm
Lincoln Index
There’s a simple statistic called the Lincoln Index that lets you estimate the total number of errors based on the number of errors found
fault-prevention  fault-removal  testing  metrics 
july 2010 by mpm
« earlier      
per page:    204080120160

Copy this bookmark:



description:


tags: