pozorvlak + testing   111

DeepState: Symbolic Unit Testing for C and C++
DeepState is a tool that provides a Google Test-like API to give C and C++ developers push-button access to symbolic execution engines, such as Manticore and angr, and fuzzers, such as Dr. Fuzz. Rather than learning multiple complex tools, users learn one interface for defining a test harness, and can use various methods to automatically generate tests for software. In addition to providing a familiar interface to binary analysis and fuzzing for parameterized unit testing, DeepState also provides constructs that aid in the construction of API-sequence tests, where the tool chooses the functions or methods to call, allowing for even more diverse and powerful tests. By serving as a front-end to multiple tools, DeepState additionally provides a way to apply (novel) high-level strategies to test generation, and to compare effectiveness and efficiency of testing back-ends, including binary analysis tools.
computers  programming  testing  c  c++  fuzzers 
2 days ago by pozorvlak
Notes on test coverage - jml :: Jonathan Lange
every new project should have 100% test coverage
every existing project should have a ratchet that enforces increasing coverage
“100% coverage” means that every line is either:
covered by the test suite
or has some markup in code saying that it is explicitly not covered, and why that’s the case
these should be enforced in CI
computers  programming  haskell  testing  continuousintegration 
october 2018 by pozorvlak
Using a Jenkinsfile
Creating a Jenkinsfile, which is checked into source control [1], provides a number of immediate benefits:

Code review/iteration on the Pipeline

Audit trail for the Pipeline

Single source of truth [2] for the Pipeline, which can be viewed and edited by multiple members of the project.
computers  programming  testing  continuousintegration 
september 2018 by pozorvlak
Packaging a python library | ionel's codelog

Put code in src.
Use tox and detox.
Test both with coverage measurements and without.
Use a generator script for tox.ini and .travis.ini.
Run the tests in Travis with tox to keep things consistent with local testing.

Too complicated? Just use a python package template.
computers  python  programming  testing  continuousintegration  packagemanagers 
september 2018 by pozorvlak
Gerald Weinberg's Secrets of Writing and Consulting: Fuzz Testing and Fuzz History
The "trash deck technique" (using decks of punch cards salvaged from the trash as test input) was universal among programmers in the 1950s.
computers  programming  fuzzers  testing  history 
september 2018 by pozorvlak
Open Jail - The Jailer Project Web Site
Jailer is a tool for database subsetting, schema and data browsing. It exports consistent, referentially intact row-sets from relational databases. It removes obsolete data without violating integrity. It is DBMS agnostic (by using JDBC), platform independent, and generates DbUnit datasets, hierarchically structured XML, and topologically sorted SQL-DML.
computers  programming  java  databases  tools  testing 
september 2018 by pozorvlak
Evaluating Fuzz Testing - 1808.09700.pdf
Most empirical analyses of fuzz-testing have bad experimental design and/or statistical analysis.
computers  testing  security  infosec  science  badscience  statistics  fuzzers 
september 2018 by pozorvlak
Linux is not about choice [was Re: Fedora too cutting edge?]
The complaints up-thread about juju and pulse are entirely valid, but the solution is not to try to deliver two things at once. If you try to deliver both at once you have to also deliver a way of switching between the two. Now you have three moving parts instead of one, which means the failure rate has gone up by a factor of _six_ (three parts, and three interactions).
linux  testing  opensource 
june 2018 by pozorvlak
Verification, coverage and maximization: The big picture – The Foretellix Blog
How hardware people think about coverage-driven verification (using coverage of an abstract state-space, specified in a verification language like e)
computers  programming  testing  hardware  fuzzers 
april 2018 by pozorvlak
Just a Whole Bunch of Different Tests • Hillel Wayne
I’ve been working on a bunch of longform obligation pieces and while they’re a lot of fun, they’re also steadily driving me insane. So I took a day off to write about all of the kinds of automated testing I know about. I’m defining tests here to be “an independent verification program that, as part of verification, executes the code we want to verify.” This means types are not tests, as they don’t involve execution of the code, and contracts are not tests, because they’re not execute...
computers  programming  testing 
april 2018 by pozorvlak
To Do Damage? (TDD) – Medium
Types qualify, document and codify the concepts chosen by the
developers in solving a problem. Testing at various levels can
certainly do some of that to different degrees but fundamentally is
about verifying that the code solves the problem in hand and has
little to say about the qualities of the concepts chosen.
computers  programming  testing  types 
january 2017 by pozorvlak
Integrated vs type based shrinking - Hypothesis
"Specifically, the way shrinking is handled in QuickCheck is bad and the way it works in Hypothesis is good."
computers  programming  testing  types 
december 2016 by pozorvlak
Quality Assured: What It’s Really Like To Test Games For A Living
RT @cynixy: All of the QA testers I know are sharing this article about what gametesting is really like:
games  testing  jobs  techindustry 
april 2016 by pozorvlak
Yeller - Getting to Zero Exceptions
Fix new unique exceptions as soon as they show up
Expected exceptions get turned into metrics
Add a regression test every time you fix an exception
Start with the most painful exceptions
programming  computers  testing  devops 
february 2016 by pozorvlak
Grenade - OpenStack
Grenade is a test harness to exercise the OpenStack upgrade process between releases. It uses DevStack to perform the initial OpenStack install and then walks through upgrading each enabled project.
openstack  testing 
january 2016 by pozorvlak
Testing v. informal reasoning
"Testing may not be sufficient to find all bugs, but it can get better reliability than nearly any company cares to"
computers  programming  testing  comparch 
december 2015 by pozorvlak
Simplifying and Isolating Failure-Inducing Input | the morning paper
RT @adriancolyer: How to minimize a failing test case to give you a much better chance of finding the problem
cs  testing  fuzzers  computers  programming  compilers 
november 2015 by pozorvlak
Test MongoDB Failure Scenarios With MockupDB
"Black pipe" testing - like black box testing, but for things with inputs and outputs.
python  testing  nosql  computers  programming 
november 2015 by pozorvlak
« earlier      
per page:    204080120160

Copy this bookmark: