cdzombak + testing   100

google/wycheproof: Project Wycheproof tests crypto libraries against known attacks.
Project Wycheproof tests crypto libraries against known attacks. It is developed and maintained by members of Google Security Team, but it is not an official Google product.

At Google, we rely on many third party cryptographic software libraries. Unfortunately, in cryptography, subtle mistakes can have catastrophic consequences, and we found that libraries fall into such implementation pitfalls much too often and for much too long. Good implementation guidelines, however, are hard to come by: understanding how to implement cryptography securely requires digesting decades' worth of academic literature. We recognize that software engineers fix and prevent bugs with unit testing, and we found that cryptographic loopholes can be resolved by the same means.

These observations have prompted us to develop Project Wycheproof, a collection of unit tests that detect known weaknesses or check for expected behaviors of some cryptographic algorithm. Project Wycheproof provides tests for most cryptographic algorithms, including RSA, elliptic curve crypto and authenticated encryption. Our cryptographers have systematically surveyed the literature and implemented most known attacks. We have over 80 test cases which have uncovered more than 40 bugs. For example, we found that we could recover the private key of widely-used DSA and ECDHC implementations.

While we are committed to develop as many attacks as possible, Project Wycheproof is by no means complete. Passing the tests does not imply that the library is secure, it just means that it is not vulnerable to the attacks that Project Wycheproof tests for. Cryptographers are also constantly discovering new attacks. Nevertheless, with Project Wycheproof developers and users now can check their libraries against a large number of known attacks, without having to spend years reading academic papers or become cryptographers themselves.
security  testing  cryptography  github_repo  lang:java 
29 days ago by cdzombak
Personal Unit Testing Road Blocks | chemistcodingios
Notes on someone's unit testing journey, in response to Fatal Error S1E9.
6 weeks ago by cdzombak
How to Do XCTestCase tearDown Wrong (and Right)
Let’s boil this down to a rule-of-thumb:

Every object you create in setUp should be destroyed in tearDown.

Remember, that’s not the only thing to do in tearDown. It’s there so we can reset things back to a clean state. What’s part of our global state? This includes:

• Method swizzling
• File system
• User defaults
• Database
• Keychain
• Really, any sort of Dependency Injection via Ambient Context
testing  xctest  lifecycle  XCTestCase  memorymanagement 
11 weeks ago by cdzombak
Test Doubles: Mocks, Stubs, and More ·
Nice introduction to different types of test doubles, plus key concepts like DI and "don't mock what you don't own".
testing  testdoubles  mocking  stubbing  documentation  ocmock  ocmockito  dependency-injection 
august 2016 by cdzombak
Behavior-Driven Development ·
As the first word in BDD suggests, you should no longer focus on tests, but you should instead focus on behaviors. This seemingly meaningless change provides an exact answer to the aforementioned question: you should test behaviors.
testing  bdd 
august 2016 by cdzombak
A Response to “Why Most Unit Testing is Waste” | Henrik Warne's blog
In my experience, unit tests are most valuable when you use them for algorithmic logic. They are not particularly useful for code that is more coordinating in its nature. Coordinating code often requires a lot of (mocked) context for unit testing, but the tests themselves are not very interesting. This type of code does not benefit a great deal from unit testing, and is instead better tested in integration testing.

also, the author who wrote, "why most unit testing is a waste" weighs in, in the comments.
august 2016 by cdzombak
MVVM Architecture Question · Issue #138 · thoughtbot/Tropos
A comment describing the use of MVVM architecture in a given application.
The whole point is to move as much code and logic out from anything View specific. This makes that code much easier to unit test. It can also lead to reusable Controllers and ViewModels.
testing  mvvm  bestpractices  software_architecture 
august 2016 by cdzombak
ashfurrow/C-41: C-41 is an application to help people develop film at home by providing a series of "recipes" for photographers to use.
C-41 is an application to help people develop film at home by providing a series of "recipes" for photographers to use.

The app demonstrates how to use Core Data, unit tests, ReactiveCocoa, and Model-View-ViewModel. Pull requests are welcome.
github_repo  coredata  software_architecture  reactivecocoa  mvvm  testing 
august 2016 by cdzombak
venmo/DVR: Network testing for Swift
DVR is a simple Swift framework for making fake NSURLSession requests for iOS, watchOS, and OS X based on VCR.

Easy dependency injection is the main design goal. The API is the same as NSURLSession. DVR.Session is a subclass of NSURLSession so you can use it as a drop in replacement anywhere. (Currently only data tasks are supported.)
ios  nsurlsession  networking  testing  github_repo 
july 2016 by cdzombak
Network Testing in Swift with DVR — Hi, I’m Sam
DVR is a project I recently wrote at work. It’s heavily inspired by VCR for Ruby. There’s a few key differences though.

Unlike VCR or other Objective-C implementations, doesn't have any global state and doesn't swizzle or monkey-path things to get in place. It's intended to by used by dependency injection.

DVR provides an NSURLSession subclass. When you try to make a request with the session, it will look for a prerecorded session. If one is found, it will play it back. If not, it will record it and then let you know you need to add the recording to your project.
testing  networking  nsurlsession  ios 
july 2016 by cdzombak
Snapshot Tests + Blinking Cursors — Medium
Nothing in the test class seemed amiss. The build server was functioning normally. The test passed on my machine. Ready to give up, hours later, it finally dawned on me.
The cursor. The damn blinking cursor.
ios  testing  uitextfield  ci  ui_testing  screenshots 
july 2016 by cdzombak
Quick/ at master · Quick/Quick
Whether you're using XCTest, Quick, or another testing framework, you can write effective unit tests by following a simple pattern:

1. Arrange
2. Act
3. Assert
testing  bestpractices  xctest  bdd  tdd 
june 2016 by cdzombak
Snapshot Testing ·
View-based testing means verifying that what the user sees is what you want the user to see. Doing this means being able to ensure that different versions of your views, or different states of your views, continue to look the same. View-based testing can be used to provide a high-level test covering a lot of use cases surrounding an object.

How It Works

FBSnapShotTestCase takes a UIView or CALayer subclass and renders it to a UIImage. This snapshot is used to create tests that compare a saved snapshot of the view/layer and the version generated by your test. When it fails, it will create a reference image of the failed test, and another image to show the difference of the two.
ios  testing  uikit  cocoa_touch  FBSnapShotTestCase 
june 2016 by cdzombak
Better Specs { rspec guidelines with ruby }
On the web there are many resources that give complete overview of _what_ you can do with RSpec. But there are fewer resources devoted to how to create a great RSpec test suite.

Better Specs tries to fill this gap by collecting most of the "best practices" that other developers learn through years of experience.
rspec  testing  bdd  documentation  bestpractices 
june 2016 by cdzombak
Home · testdouble/contributing-tests Wiki
This wiki presents some notes on testing.

That's an understatement. This looks to be a very solid resource on TDD.
github_wiki  testing  tdd  bdd  bestpractices  documentation 
june 2016 by cdzombak
describe vs. context in rspec – Lab Matrix
The purpose of “describe” is to wrap a set of tests against one functionality while “context” is to wrap a set of tests against one functionality under the same state.
testing  rspec  bdd 
may 2016 by cdzombak
5 Questions Every Unit Test Must Answer — JavaScript Scene — Medium
Lots of good advice about unit testing and TDD in here. This is not at all JS-specific.
Next time you write a test, remember to answer all the questions:

• What are you testing?
• What should it do?
• What is the actual output?
• What is the expected output?
• How can the test be reproduced?
testing  tdd  bestpractices 
may 2016 by cdzombak
UI Testing in Xcode 7 ·
Good discussion on ins & outs of UI testing in Xcode.
xcode  ui  testing  xctest  accessibility 
april 2016 by cdzombak
Mocking Classes You Don't Own ·
To accomplish this we will follow my coined acronym, IDEPEM. This stands for inject dependencies, everything's a protocol, equatable mocks.
testing  swiftlang  protocols  mocking 
march 2016 by cdzombak
Transitioning From Objective C to Swift in 4 Steps – Without Rewriting The Existing Code | Code Voyagers
the most significant takeaway (aside from basic interop notes):
In Swift the only way to write testable code is by making dependencies explicit and injectable.

there is no mocking. this is not a bad thing.
swiftlang  objective-c  interop  testing  mocking  ocmock 
february 2016 by cdzombak
Partial functions in Swift, Part 2: Catching precondition failures
in which we use Mach exception handlers to rewrite precondition statements into catchable failures in order to test them properly.
mach  testing  swiftlang  precondition  assertions  error_handling  xctest 
february 2016 by cdzombak
SwiftKit/Cuckoo: First boilerplate-free mocking framework for Swift!
Cuckoo was created due to lack of a proper Swift mocking framework. We built the DSL to be very similar to Mockito, so anyone using it in Java/Android can immediately pick it up and use it.

## How does it work

Cuckoo has two parts. One is the runtime and the other one is an OS X command-line tool simply called CuckooGenerator.

Unfortunately Swift does not have a proper reflection, so we decided to use a compile-time generator to go through files you specify and generate supporting structs/classes that will be used by the runtime in your test target.

The generated files contain enough information to give you the right amount of power.
mocking  testing  swiftlang  lang:swift  ios  osx  github_repo 
january 2016 by cdzombak
KnuffApp/Knuff: The debug application for Apple Push Notification Service (APNs).
Knuff: The debug application for Apple Push Notification Service (APNs).


• Send push notifications to APNS (Apple Push Notification Service) very easily (no configuration needed at all)
• Load / Save documents including token and JSON payload
• Grabs the certificate right from your keychain
• Get the device token automatically; forget about manually retrieving the device token through logging or similar techniques. Even more useful when not in sandbox mode
• Support for error response codes
• Detects Development/Production environment automatically
• Supports universal certificates
• Custom JSON payloads
• Identity export to PEM format (⌘ + E)
apple  notifications  ios  testing  tool  apns  osx  github_repo  via:iosdevweekly 
january 2016 by cdzombak
The Big List of Naughty Strings is a list of strings which have a high probability of causing issues when used as user-input data.

I have put together these strings to create Naughty Keyboard – a very simple iOS app that gives you access to a custom keyboard to use these strings to test your application.

Big List of Naughty Strings:
testing  ios  github_repo 
august 2015 by cdzombak
The Big List of Naughty Strings is an evolving list of strings which have a high probability of causing issues when used as user-input data. This is intended for use in helping both automated and manual QA testing;

See for an iOS keyboard
testing  github_repo 
august 2015 by cdzombak
A Matcher Framework for Swift and Objective-C
ios  testing  swiftlang  objective-c  github_repo 
july 2015 by cdzombak
We've Got You Covered | Big Nerd Ranch
One of Xcode 7’s new features is in-line display of code coverage metrics. Yay? That’s a pretty dull way of describing a nice feature. Code coverage figures out which lines of code have been executed while running unit tests, and now Xcode shows you information about this coverage in the UI.
xcode  testing  llvm 
july 2015 by cdzombak
A Closer Look at Test Spies
A discussion of some tradeoffs of test spies vs. mocks, and the new-in-RSpec-3.1 `spy` method.
testing  test_spies  mocking  rspec 
june 2015 by cdzombak
Spy vs Spy
Test spies allow clean separation of test phases; and they allow reusability of stub objects without verifying the same things repeatedly.
testing  test_spies 
june 2015 by cdzombak
With the latest support library release 22.2.0, we added an @VisibleForTesting…
With the latest support library release 22.2.0, we added an @VisibleForTesting annotation to support-annotations[0]. This annotation denotes that a class, method or field has its visibility relaxed, so that it is more widely visible than otherwise necessary to make your code more testable.

Setting aside the debate around testing _only_ the public interface, this is pragmatic and would be useful. It should raise a red flag during code review, but it isn't necessarily always bad.
testing  android  java  annotations 
june 2015 by cdzombak
Quick is a behavior-driven development framework for Swift and Objective-C.
bdd  testing  objective-c  swiftlang  github_repo 
may 2015 by cdzombak
Microsoft Research Mobile - Exploding Software-Engineering Myths
After looking closely at code coverage, Nagappan now can state that, given constraints, it is more beneficial to achieve higher code coverage of more complex code than to test less complex code at an equivalent level. Those are the kinds of tradeoffs that development managers need to keep in mind.
What the research team found was that the TDD teams produced code that was 60 to 90 percent better in terms of defect density than non-TDD teams. They also discovered that TDD teams took longer to complete their projects—15 to 35 percent longer.

“Over a development cycle of 12 months, 35 percent is another four months, which is huge,” Nagappan says. “However, the tradeoff is that you reduce post-release maintenance costs significantly, since code quality is so much better. Again, these are decisions that managers have to make—where should they take the hit? But now, they actually have quantified data for making those decisions.”
The team observed a definite negative correlation: more assertions and code verifications means fewer bugs. Looking behind the straight statistical evidence, they also found a contextual variable: experience. Software engineers who were able to make productive use of assertions in their code base tended to be well-trained and experienced, a factor that contributed to the end results. These factors built an empirical body of knowledge that proved the utility of assertions.
Organizational metrics, which are not related to the code, can predict software failure-proneness with a precision and recall of 85 percent.
…they looked for statistical evidence that components developed by distributed teams resulted in software with more errors than components developed by collocated teams.

Does distributed development affect software quality? An empirical case study of Windows Vista—by Christian Bird, University of California, Davis; Nagappan; Premkumar Devanbu, University of California, Davis; Harald Gall, University of Zurich, and Murphy—found that the differences were statistically negligible. In order to verify results, the team also conducted an anonymous survey with researchers Sriram Rajamani and Ganesan Ramalingam in Microsoft Research India, asking engineers who they would talk to if they ran into problems. Most people preferred to talk to someone from their own organization 4,000 miles away rather than someone only five doors down the hall but from a different organization. Organizational cohesiveness played a bigger role than geographical distance.

Also contains links to the relevant studies.
programming  testing  tdd  productivity  management  remote_work  work 
march 2015 by cdzombak
Using Overridable Properties Instead of "Tell Don't Ask" | iOS Unit Testing
u:andrewsardone first turned me on to this approach, which I strongly believe is usually the right choice. It's not _not_ Tell Don't Ask, but I'd call it…"Ask If Not Told Otherwise".
testing  dependency-injection  designpatterns  tell-dont-ask  oop 
february 2015 by cdzombak
Tool to test that static sites (e.g. a Jekyll blog) are not broken: link references, img srcs, images have alt attributes etc.
jekyll  octopress  html  testing  validation  via:github  via:henrik 
january 2015 by cdzombak
Testing View Controllers - Lighter View Controllers - issue #1
Covers, among other things, mocking view classes. Not sure how much I agree with this, esp. when using MVVM.
objective-c  cocoa_touch  testing  sentestkit  ocmock  uiviewcontroller 
december 2014 by cdzombak
Asynchronous Unit Testing in Xcode 6 | dada beatnik
Apple have delivered a means to implement asynchronous unit tests in an intelligent and offcially supported way.
xcode  xcode6  xctest  XCTestExpectation  testing  async 
october 2014 by cdzombak
Singletons are Pathological Liars
Singletons are nothing more than global state. Global state makes it so your objects can secretly get hold of things which are not declared in their APIs, and, as a result, Singletons make your APIs into pathological liars.
singleton  testing  via:andrewsardone  designpatterns  antipatterns  oop 
october 2014 by cdzombak
Mocks Aren't Stubs
Meszaros uses the term Test Double as the generic term for any kind of pretend object used in place of a real object for testing purposes. The name comes from the notion of a Stunt Double in movies. (One of his aims was to avoid using any name that was already widely used.) Meszaros then defined four particular kinds of double:

Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.

Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an in memory database is a good example).

Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it 'sent', or maybe only how many messages it 'sent'.

Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.

Of these kinds of doubles, only mocks insist upon behavior verification. The other doubles can, and usually do, use state verification. Mocks actually do behave like other doubles during the exercise phase, as they need to make the SUT believe it's talking with its real collaborators - but mocks differ in the setup and the verification phases.
tdd  testing  goos  oop 
august 2014 by cdzombak
Test Isolation Is About Avoiding Mocks — Destroy All Software Blog
Gary Bernhardt's second post on the straw-men set up by those suggesting that isolated testing and/or “mocks” destroy your program’s design.
Deeply nested mocks tell us little about mocks, just as deeply nested conditionals tell us little about structured programming. Proficient users of structured programming rarely write deeply nested conditionals; proficient users of mocks rarely write deeply nested mocks.

For notes that directly apply to DHH, see GB’s [first post on TDD rhetoric][1].


## cdzombak added:
Isolated testing has an easily identified villain: the deeply-nested mock object.

A big part of this seems to be that structuring your program well (cf. Law of Demeter) leads to better tests, and that deeply nested mocks indicate violation of the principle of least knowledge.
The cost of isolating complex code counteracts the desire to be lazy and avoid extracting a new method or a new class.
mockobjects  OOP  goos  isolation  tdd  testing  mocking  via:andrewsardone 
august 2014 by cdzombak
XCTestCase / XCTestExpectation / measureBlock() - NSHipster
With Xcode 6, we've finally arrived: the built-in testing tools are now good enough to use on their own. That is to say, there are no particularly compelling reasons to use any additional abstractions in order to provide acceptable test coverage for the vast majority apps and libraries. Except in extreme cases that require extensive stubbing, mocking, or other exotic test constructs, XCTest assertions, expectations, and performance measurements should be sufficient.

mocking  testing  xctest  xcode  swiftlang 
july 2014 by cdzombak
facebook/ios-snapshot-test-case - Snapshot view unit tests for iOS
A "snapshot test case" takes a configured UIView or CALayer and uses the renderInContext: method to get an image snapshot of its contents. It compares this snapshot to a "reference image" stored in your source code repository and fails the test if the two images don't match.
ios  cocoa_touch  testing  via:osteslag  uikit  FBSnapShotTestCase 
july 2014 by cdzombak
Unit Testing Asynchronous Cocoa
Testing asynchronous Cocoa APIs using the runloop.
The way I run asynchronous operations in unit tests for Ensembles is to include two methods in the test class. The first starts the run loop, and does not return until the run loop is stopped.

- (void)waitForAsynchronousTask

The second method explicitly stops the run loop, causing the first method to return.

- (void)completeAsynchronousTask

The first method is called at the end of the test method, and the second method is called inside the completion handler of the code being tested.
cocoa  testing  objective-c  via:brentsimmons  runloop 
may 2014 by cdzombak
Mock Roles, not Objects
Our understanding of the technique has deepened, in particular we now have a much stronger bias towards using Mock Objects for design, rather than just testing. We now understand its role in driving good design from requirements, and its technical limitations. We have embodied our experience in jMock, a new
generation of Mock Object framework that we believe gives us the expressiveness we need to support Need-Driven Development.
tdd  testing  filetype:pdf  software_architecture  mocking  oop 
march 2014 by cdzombak
Michael Feathers: The Framework Superclass Anti-Pattern
If you are developing frameworks do not provide superclasses that framework users must inherit to use your framework. Inheritance is the one of the tightest forms of coupling you can use in OO. When you force your users to inherit from you, you:

• Make it nearly impossible for users to test their logic independently of your framework
• Make migration away from your framework difficult, or impossible.
software_architecture  designpatterns  inheritance  antipatterns  testing  oop 
january 2014 by cdzombak
When to fake; when to mock - The Code Whisperer
1. Am I using the objects around me correctly? Am I invoking the right methods at the right times with the right parameters?

2. Am I reacting to the objects around me correctly? Do I handle errors reasonably? Do I respond well to the answers they send me?

For the first kind of test, I generally mock; and for the second kind of test, I generally fake.

Put another way, I lean on interaction-based testing to verify how my object talks to its collaborators; but I lean on state-based testing to verify how well that object listens.
december 2013 by cdzombak
When is it safe to introduce test doubles? - The Code Whisperer
Never mock values, sometimes mock entities, but mock services freely.

You should have no test doubles for values. If you do, it’s time to uncover what mislaid responsibility your value has, extract it out, then try again.

You might want to mock an entity, but more likely you want to mock how you got the entity.

Services, you can mock with impunity. In fact, the only time not to mock a service is when you’re testing the service. Since services are typically interfaces and pluggable, you’ll want to use test doubles when you test how the application uses the service, or how a service uses another service.
december 2013 by cdzombak
« earlier      
per page:    204080120160

related tags

accessibility  android  annotations  antipatterns  apns  apple  appstore  aspectoriented  assertions  async  bdd  benchmarking  bestpractices  ci  circuit_breaker  cocoa  cocoa_touch  code_coverage  coredata  cryptography  data  dataset  dependency-injection  designpatterns  development  diy  documentation  elixir  embedded  erlang  error_handling  expecta  faulttolerance  FBSnapShotTestCase  filetype:pdf  github_repo  github_wiki  goos  html  iap  inheritance  interop  ios  isolation  itunesconnect  java  javascript  jekyll  jenkins  lang:java  lang:swift  lifecycle  llvm  mach  management  memorymanagement  mocking  mockito  mockobjects  mvvm  names  networking  nimble  notifications  NSErrorPointer  NSURLConnection  nsurlsession  objective-c  ocmock  ocmockito  octopress  oop  osx  pc  php  precondition  productivity  programming  protocols  quick  quickcheck  rails  reactivecocoa  remote_work  robolectric  rspec  ruby  runloop  scala  screenshots  security  semian  sentestkit  singleton  software_architecture  stability  stubbing  swiftlang  tdd  tell-dont-ask  testdoubles  testing  test_spies  tool  tools  toyota  travisci  ui  UIAutomation  uikit  uitesting  uitextfield  uiviewcontroller  ui_testing  validation  via:andrewsardone  via:brentsimmons  via:github  via:henrik  via:iosdevweekly  via:mattb  via:mikelinington  via:osteslag  web_dev  work  xcode  xcode6  xcode7  xcodebots  xctest  XCTestCase  XCTestExpectation 

Copy this bookmark: