cdzombak + testing   110

The Joel Test: 12 Steps to Better Code – Joel on Software
Do you use source control?
Can you make a build in one step?
Do you make daily builds?
Do you have a bug database?
Do you fix bugs before writing new code?
Do you have an up-to-date schedule?
Do you have a spec?
Do programmers have quiet working conditions?
Do you use the best tools money can buy?
Do you have testers?
Do new candidates write code during their interview?
Do you do hallway usability testing?
programming  bestpractices  culture  testing  interviewing  management 
18 days ago by cdzombak
Don't mock types you don't own - dave^2 = -1
In summary, my current thinking on the subject is that while using mocks and stubs is really beneficial when driving out the design of collaborators using TDD, it is important to identify when we need to stop test-driving and start testing. This is generally the point where we hit a type we don’t own from a library or external system.

Using test doubles for types we don’t own can end up with fragile tests that don’t actually test much of value, or can even compromise our design and the effectiveness of the abstractions we use. Even when mocking a library we know really well we can end up with compromised abstractions and fragile tests due to relying on implementation details or assumptions based on previous versions of the library.

I’ve found switching to integration and acceptance tests to test over the boundary between my code and the other types a very useful alternative, but it is important to be aware that this approach can bring its own problems, and we need to try and mitigate these when writing our tests.

And finally, we should remember that this is just a guideline. If we find a situation where it is going to be both safe and simpler to mock a type we don’t own, we may as well do so, but at least we’ve considered our options. :)
testing  tdd  mocking  bestpractices 
21 days ago by cdzombak
TDD: Only mock types you own - Mark Needham at Mark Needham
There are a couple of reasons why doing this [mocking types you don't own, for interaction testing] isn’t a great idea:

• We have no idea what the correct method calls are in the first place so we’re just guessing based on looking through the Hibernate code and selecting the methods that we think make it work correctly.
• If the library code gets changed then our tests break even though functionally the code might still work
mocking  testing  tdd  bestpractices 
22 days ago by cdzombak
Mock Objects | Liz Douglass
This morning I was reading Mock Roles, Not Objects and I was was reminded of my tennis experiences. It refreshed my thinking on the purpose and practices of TDD and mocking. There were several quotes from the paper that stood out for me:

• “Using TDD has many benefits but the most relevant is that it directs the programmer to think about the design of code from its intended use, rather from its implementation.”
• “Mock Objects changes the focus of TDD from thinking about the changes in state of an object to thinking about its interactions with other objects.”
• “Writing tests drives a framework to think about functionality. Mock Objects provides a framework for making assertions about those relationships and for simulating responses.”

Section 4 of the paper discusses Mock Objects in practices, detailing some guidelines for how to use them properly:

• “Only mock types you own”. Matt mentioned this one at our book club the other day in the context of skinning and wrapping APIs.
• “Don’t use getters”. Why? “Getters expose implementation, which increases the coupling between objects and allows responsibilities to be left in the wrong module. Avoiding getters forces an emphasis on object behaviour rather than state..”. I was taught not to use getters by Nick and I find myself discussing it often with each pair.
• “Specify as little as possible in a test”…”One of the risks with TDD is that tests become ‘brittle’, that is they fail when a programmer makes unrelated changes to application code.” This is on of the most common criticisms of TDD that I have heard from other developers. The paper goes onto say that the cause is that “…(the tests) have been over-specfied to check features that are an artefact of implementation, not an expression of some requirement in the object.” I’ve written tests that fall into in this category and I think it can be quite challenging to avoid doing so. Steve Freeman, one of the authors of the paper, blogged about this situation happening in a Coding Dojo earlier this year.
mocking  testing  bestpractices  tdd 
22 days ago by cdzombak
Mock Roles, not Objects
Mock Objects is an extension to Test-Driven Development that supports good Object-Oriented design by guiding the discovery of a coherent system of types within a code base. It turns out to be less interesting as a technique for isolating tests from third-party libraries than is widely thought. This paper describes the process of using Mock Objects with an extended example and reports best and worst practices gained from experience of applying the process. It also introduces jMock, a Java framework that embodies our collective experience.
filetype:pdf  testing  tdd  bestpractices  mocking 
22 days ago by cdzombak
google/wycheproof: Project Wycheproof tests crypto libraries against known attacks.
Project Wycheproof tests crypto libraries against known attacks. It is developed and maintained by members of Google Security Team, but it is not an official Google product.

At Google, we rely on many third party cryptographic software libraries. Unfortunately, in cryptography, subtle mistakes can have catastrophic consequences, and we found that libraries fall into such implementation pitfalls much too often and for much too long. Good implementation guidelines, however, are hard to come by: understanding how to implement cryptography securely requires digesting decades' worth of academic literature. We recognize that software engineers fix and prevent bugs with unit testing, and we found that cryptographic loopholes can be resolved by the same means.

These observations have prompted us to develop Project Wycheproof, a collection of unit tests that detect known weaknesses or check for expected behaviors of some cryptographic algorithm. Project Wycheproof provides tests for most cryptographic algorithms, including RSA, elliptic curve crypto and authenticated encryption. Our cryptographers have systematically surveyed the literature and implemented most known attacks. We have over 80 test cases which have uncovered more than 40 bugs. For example, we found that we could recover the private key of widely-used DSA and ECDHC implementations.

While we are committed to develop as many attacks as possible, Project Wycheproof is by no means complete. Passing the tests does not imply that the library is secure, it just means that it is not vulnerable to the attacks that Project Wycheproof tests for. Cryptographers are also constantly discovering new attacks. Nevertheless, with Project Wycheproof developers and users now can check their libraries against a large number of known attacks, without having to spend years reading academic papers or become cryptographers themselves.
security  testing  cryptography  github_repo  lang:java 
9 weeks ago by cdzombak
Personal Unit Testing Road Blocks | chemistcodingios
Notes on someone's unit testing journey, in response to Fatal Error S1E9.
testing 
11 weeks ago by cdzombak
How to Do XCTestCase tearDown Wrong (and Right)
Let’s boil this down to a rule-of-thumb:

Every object you create in setUp should be destroyed in tearDown.

Remember, that’s not the only thing to do in tearDown. It’s there so we can reset things back to a clean state. What’s part of our global state? This includes:

• Method swizzling
• File system
• User defaults
• Database
• Keychain
• Really, any sort of Dependency Injection via Ambient Context
testing  xctest  lifecycle  XCTestCase  memorymanagement 
october 2016 by cdzombak
Test Doubles: Mocks, Stubs, and More · objc.io
Nice introduction to different types of test doubles, plus key concepts like DI and "don't mock what you don't own".
testing  testdoubles  mocking  stubbing  documentation  ocmock  ocmockito  dependency-injection 
august 2016 by cdzombak
Behavior-Driven Development · objc.io
As the first word in BDD suggests, you should no longer focus on tests, but you should instead focus on behaviors. This seemingly meaningless change provides an exact answer to the aforementioned question: you should test behaviors.
testing  bdd 
august 2016 by cdzombak
A Response to “Why Most Unit Testing is Waste” | Henrik Warne's blog
In my experience, unit tests are most valuable when you use them for algorithmic logic. They are not particularly useful for code that is more coordinating in its nature. Coordinating code often requires a lot of (mocked) context for unit testing, but the tests themselves are not very interesting. This type of code does not benefit a great deal from unit testing, and is instead better tested in integration testing.


also, the author who wrote, "why most unit testing is a waste" weighs in, in the comments.
testing 
august 2016 by cdzombak
MVVM Architecture Question · Issue #138 · thoughtbot/Tropos
A comment describing the use of MVVM architecture in a given application.
The whole point is to move as much code and logic out from anything View specific. This makes that code much easier to unit test. It can also lead to reusable Controllers and ViewModels.
testing  mvvm  bestpractices  software_architecture 
august 2016 by cdzombak
ashfurrow/C-41: C-41 is an application to help people develop film at home by providing a series of "recipes" for photographers to use.
C-41 is an application to help people develop film at home by providing a series of "recipes" for photographers to use.

The app demonstrates how to use Core Data, unit tests, ReactiveCocoa, and Model-View-ViewModel. Pull requests are welcome.
github_repo  coredata  software_architecture  reactivecocoa  mvvm  testing 
august 2016 by cdzombak
venmo/DVR: Network testing for Swift
DVR is a simple Swift framework for making fake NSURLSession requests for iOS, watchOS, and OS X based on VCR.

Easy dependency injection is the main design goal. The API is the same as NSURLSession. DVR.Session is a subclass of NSURLSession so you can use it as a drop in replacement anywhere. (Currently only data tasks are supported.)
ios  nsurlsession  networking  testing  github_repo 
july 2016 by cdzombak
Network Testing in Swift with DVR — Hi, I’m Sam
DVR is a project I recently wrote at work. It’s heavily inspired by VCR for Ruby. There’s a few key differences though.

Unlike VCR or other Objective-C implementations, doesn't have any global state and doesn't swizzle or monkey-path things to get in place. It's intended to by used by dependency injection.

DVR provides an NSURLSession subclass. When you try to make a request with the session, it will look for a prerecorded session. If one is found, it will play it back. If not, it will record it and then let you know you need to add the recording to your project.
testing  networking  nsurlsession  ios 
july 2016 by cdzombak
Snapshot Tests + Blinking Cursors — Medium
Nothing in the test class seemed amiss. The build server was functioning normally. The test passed on my machine. Ready to give up, hours later, it finally dawned on me.
The cursor. The damn blinking cursor.
ios  testing  uitextfield  ci  ui_testing  screenshots 
july 2016 by cdzombak
Quick/ArrangeActAssert.md at master · Quick/Quick
Whether you're using XCTest, Quick, or another testing framework, you can write effective unit tests by following a simple pattern:

1. Arrange
2. Act
3. Assert
testing  bestpractices  xctest  bdd  tdd 
june 2016 by cdzombak
Snapshot Testing · objc.io
View-based testing means verifying that what the user sees is what you want the user to see. Doing this means being able to ensure that different versions of your views, or different states of your views, continue to look the same. View-based testing can be used to provide a high-level test covering a lot of use cases surrounding an object.

How It Works

FBSnapShotTestCase takes a UIView or CALayer subclass and renders it to a UIImage. This snapshot is used to create tests that compare a saved snapshot of the view/layer and the version generated by your test. When it fails, it will create a reference image of the failed test, and another image to show the difference of the two.
ios  testing  uikit  cocoa_touch  FBSnapShotTestCase 
june 2016 by cdzombak
Better Specs { rspec guidelines with ruby }
On the web there are many resources that give complete overview of _what_ you can do with RSpec. But there are fewer resources devoted to how to create a great RSpec test suite.

Better Specs tries to fill this gap by collecting most of the "best practices" that other developers learn through years of experience.
rspec  testing  bdd  documentation  bestpractices 
june 2016 by cdzombak
Home · testdouble/contributing-tests Wiki
This wiki presents some notes on testing.


That's an understatement. This looks to be a very solid resource on TDD.
github_wiki  testing  tdd  bdd  bestpractices  documentation 
june 2016 by cdzombak
describe vs. context in rspec – Lab Matrix
The purpose of “describe” is to wrap a set of tests against one functionality while “context” is to wrap a set of tests against one functionality under the same state.
testing  rspec  bdd 
may 2016 by cdzombak
5 Questions Every Unit Test Must Answer — JavaScript Scene — Medium
Lots of good advice about unit testing and TDD in here. This is not at all JS-specific.
Next time you write a test, remember to answer all the questions:

• What are you testing?
• What should it do?
• What is the actual output?
• What is the expected output?
• How can the test be reproduced?
testing  tdd  bestpractices 
may 2016 by cdzombak
UI Testing in Xcode 7 · Masilotti.com
Good discussion on ins & outs of UI testing in Xcode.
xcode  ui  testing  xctest  accessibility 
april 2016 by cdzombak
Mocking Classes You Don't Own · Masilotti.com
To accomplish this we will follow my coined acronym, IDEPEM. This stands for inject dependencies, everything's a protocol, equatable mocks.
testing  swiftlang  protocols  mocking  nsurlsession 
march 2016 by cdzombak
Transitioning From Objective C to Swift in 4 Steps – Without Rewriting The Existing Code | Code Voyagers
the most significant takeaway (aside from basic interop notes):
In Swift the only way to write testable code is by making dependencies explicit and injectable.

there is no mocking. this is not a bad thing.
swiftlang  objective-c  interop  testing  mocking  ocmock 
february 2016 by cdzombak
Partial functions in Swift, Part 2: Catching precondition failures
in which we use Mach exception handlers to rewrite precondition statements into catchable failures in order to test them properly.
mach  testing  swiftlang  precondition  assertions  error_handling  xctest 
february 2016 by cdzombak
SwiftKit/Cuckoo: First boilerplate-free mocking framework for Swift!
Cuckoo was created due to lack of a proper Swift mocking framework. We built the DSL to be very similar to Mockito, so anyone using it in Java/Android can immediately pick it up and use it.

## How does it work

Cuckoo has two parts. One is the runtime and the other one is an OS X command-line tool simply called CuckooGenerator.

Unfortunately Swift does not have a proper reflection, so we decided to use a compile-time generator to go through files you specify and generate supporting structs/classes that will be used by the runtime in your test target.

The generated files contain enough information to give you the right amount of power.
mocking  testing  swiftlang  lang:swift  ios  osx  github_repo 
january 2016 by cdzombak
KnuffApp/Knuff: The debug application for Apple Push Notification Service (APNs).
Knuff: The debug application for Apple Push Notification Service (APNs).

Features:

• Send push notifications to APNS (Apple Push Notification Service) very easily (no configuration needed at all)
• Load / Save documents including token and JSON payload
• Grabs the certificate right from your keychain
• Get the device token automatically; forget about manually retrieving the device token through logging or similar techniques. Even more useful when not in sandbox mode
• Support for error response codes
• Detects Development/Production environment automatically
• Supports universal certificates
• Custom JSON payloads
• Identity export to PEM format (⌘ + E)
apple  notifications  ios  testing  tool  apns  osx  github_repo  via:iosdevweekly 
january 2016 by cdzombak
Palleas/NaughtyKeyboard
The Big List of Naughty Strings is a list of strings which have a high probability of causing issues when used as user-input data.

I have put together these strings to create Naughty Keyboard – a very simple iOS app that gives you access to a custom keyboard to use these strings to test your application.

Big List of Naughty Strings: https://pinboard.in/u:cdzombak/b:3da55bc0db36
testing  ios  github_repo 
august 2015 by cdzombak
minimaxir/big-list-of-naughty-strings
The Big List of Naughty Strings is an evolving list of strings which have a high probability of causing issues when used as user-input data. This is intended for use in helping both automated and manual QA testing;

See https://pinboard.in/u:cdzombak/b:82861dbcfd2b for an iOS keyboard
testing  github_repo 
august 2015 by cdzombak
Quick/Nimble
A Matcher Framework for Swift and Objective-C
ios  testing  swiftlang  objective-c  github_repo 
july 2015 by cdzombak
We've Got You Covered | Big Nerd Ranch
One of Xcode 7’s new features is in-line display of code coverage metrics. Yay? That’s a pretty dull way of describing a nice feature. Code coverage figures out which lines of code have been executed while running unit tests, and now Xcode shows you information about this coverage in the UI.
xcode  testing  llvm 
july 2015 by cdzombak
A Closer Look at Test Spies
A discussion of some tradeoffs of test spies vs. mocks, and the new-in-RSpec-3.1 `spy` method.
testing  test_spies  mocking  rspec 
june 2015 by cdzombak
Spy vs Spy
Test spies allow clean separation of test phases; and they allow reusability of stub objects without verifying the same things repeatedly.
testing  test_spies 
june 2015 by cdzombak
With the latest support library release 22.2.0, we added an @VisibleForTesting…
With the latest support library release 22.2.0, we added an @VisibleForTesting annotation to support-annotations[0]. This annotation denotes that a class, method or field has its visibility relaxed, so that it is more widely visible than otherwise necessary to make your code more testable.


Setting aside the debate around testing _only_ the public interface, this is pragmatic and would be useful. It should raise a red flag during code review, but it isn't necessarily always bad.
testing  android  java  annotations 
june 2015 by cdzombak
Quick/Quick
Quick is a behavior-driven development framework for Swift and Objective-C.
bdd  testing  objective-c  swiftlang  github_repo 
may 2015 by cdzombak
Microsoft Research Mobile - Exploding Software-Engineering Myths
After looking closely at code coverage, Nagappan now can state that, given constraints, it is more beneficial to achieve higher code coverage of more complex code than to test less complex code at an equivalent level. Those are the kinds of tradeoffs that development managers need to keep in mind.
What the research team found was that the TDD teams produced code that was 60 to 90 percent better in terms of defect density than non-TDD teams. They also discovered that TDD teams took longer to complete their projects—15 to 35 percent longer.

“Over a development cycle of 12 months, 35 percent is another four months, which is huge,” Nagappan says. “However, the tradeoff is that you reduce post-release maintenance costs significantly, since code quality is so much better. Again, these are decisions that managers have to make—where should they take the hit? But now, they actually have quantified data for making those decisions.”
The team observed a definite negative correlation: more assertions and code verifications means fewer bugs. Looking behind the straight statistical evidence, they also found a contextual variable: experience. Software engineers who were able to make productive use of assertions in their code base tended to be well-trained and experienced, a factor that contributed to the end results. These factors built an empirical body of knowledge that proved the utility of assertions.
Organizational metrics, which are not related to the code, can predict software failure-proneness with a precision and recall of 85 percent.
…they looked for statistical evidence that components developed by distributed teams resulted in software with more errors than components developed by collocated teams.

Does distributed development affect software quality? An empirical case study of Windows Vista—by Christian Bird, University of California, Davis; Nagappan; Premkumar Devanbu, University of California, Davis; Harald Gall, University of Zurich, and Murphy—found that the differences were statistically negligible. In order to verify results, the team also conducted an anonymous survey with researchers Sriram Rajamani and Ganesan Ramalingam in Microsoft Research India, asking engineers who they would talk to if they ran into problems. Most people preferred to talk to someone from their own organization 4,000 miles away rather than someone only five doors down the hall but from a different organization. Organizational cohesiveness played a bigger role than geographical distance.

Also contains links to the relevant studies.
programming  testing  tdd  productivity  management  remote_work  work 
march 2015 by cdzombak
Using Overridable Properties Instead of "Tell Don't Ask" | iOS Unit Testing
u:andrewsardone first turned me on to this approach, which I strongly believe is usually the right choice. It's not _not_ Tell Don't Ask, but I'd call it…"Ask If Not Told Otherwise".
testing  dependency-injection  designpatterns  tell-dont-ask  oop 
february 2015 by cdzombak
gjtorikian/html-proofer
Tool to test that static sites (e.g. a Jekyll blog) are not broken: link references, img srcs, images have alt attributes etc.
jekyll  octopress  html  testing  validation  via:github  via:henrik 
january 2015 by cdzombak
Testing View Controllers - Lighter View Controllers - objc.io issue #1
Covers, among other things, mocking view classes. Not sure how much I agree with this, esp. when using MVVM.
objective-c  cocoa_touch  testing  sentestkit  ocmock  uiviewcontroller 
december 2014 by cdzombak
Asynchronous Unit Testing in Xcode 6 | dada beatnik
Apple have delivered a means to implement asynchronous unit tests in an intelligent and offcially supported way.
xcode  xcode6  xctest  XCTestExpectation  testing  async 
october 2014 by cdzombak
Singletons are Pathological Liars
Singletons are nothing more than global state. Global state makes it so your objects can secretly get hold of things which are not declared in their APIs, and, as a result, Singletons make your APIs into pathological liars.
singleton  testing  via:andrewsardone  designpatterns  antipatterns  oop 
october 2014 by cdzombak
« earlier      
per page:    204080120160

related tags

accessibility  android  annotations  antipatterns  apns  apple  appstore  aspectoriented  assertions  async  bdd  benchmarking  bestpractices  ci  circuit_breaker  cocoa  cocoa_touch  code_coverage  coredata  cryptography  culture  data  dataset  dependency-injection  designpatterns  development  diy  documentation  elixir  embedded  erlang  error_handling  expecta  faulttolerance  FBSnapShotTestCase  filetype:pdf  github_repo  github_wiki  goos  html  iap  inheritance  interop  interviewing  ios  isolation  itunesconnect  java  javascript  jekyll  jenkins  lang:java  lang:swift  lifecycle  llvm  mach  management  memorymanagement  mocking  mockito  mockobjects  mvvm  names  networking  nimble  notifications  NSErrorPointer  NSURLConnection  nsurlsession  objective-c  ocmock  ocmockito  octopress  oop  osx  pc  php  precondition  productivity  programming  protocols  quick  quickcheck  rails  reactivecocoa  remote_work  robolectric  rspec  ruby  runloop  scala  screenshots  security  semian  sentestkit  singleton  software_architecture  stability  stubbing  swiftlang  tdd  tell-dont-ask  testdoubles  testing  test_spies  tool  tools  toyota  travisci  ui  UIAutomation  uikit  uitesting  uitextfield  uiviewcontroller  ui_testing  validation  via:andrewsardone  via:brentsimmons  via:github  via:henrik  via:iosdevweekly  via:mattb  via:mikelinington  via:osteslag  web_dev  work  xcode  xcode6  xcode7  xcodebots  xctest  XCTestCase  XCTestExpectation  XCTestWaiter  XCTKVOExpectation  XCTNSNotificationExpectation  XCTPredicateExpectation 

Copy this bookmark:



description:


tags: