NeoVim and Testing

Oct 14, 2017 09:34 · 1023 words · 5 minutes read Python Testing Vim

One of the cool things you can do with most modern IDEs is have tests continually run while you are editing code and see the effects. Bonus points if the IDE has tooling that supports only running test cases that are effected by the latest edits. You can approximate this effect by running pytest-watch In a separate terminal or in a tmux split, but it’s not quite the same.

I recently spent a little time improving the tooling for my NeoVim environment and my two big goals were

  1. Be able to run unit tests continuously in a NeoVim buffer
  2. Be able to show code coverage of a file in NeoVim

My focus is specifically for python, but you could actually achieve the same result with any language supported by the plugins used.

Running tests in NeoVim

This is pretty easy to accomplish with the excellent vim-test plugin. There’s some configuration needed to get this to work with pytest but the documentation makes it very straight forward.

" Contents of init.vim
call plug#begin(~/.vim/plugged')
Plug 'janko-m/vim-test
call plug#end()

" Binding for running individual files
nnoremap <leader>tf :TestFile<cr>

With this setup we can open a test file and type <leader>tf which will run the test file through pytest (assuming it’s installed in your virtualenv). That’s cool but we’d really like to keep running pytest every time the file changes, how can we accomplish that?

pytest-watch

vim-test has a way to change the executable when running the test commands. We can make one minor change to the contents of init.vim

" Contents of init.vim

let test#python#pytest#executable = 'ptw --'

Which will allow us to run pytest-watch in place of pytest. Running <leader>tf again will present a problem. The buffer opens and pytest-watch runs correctly but we get trapped in that buffer and can’t return to editing until we ^C out.

split-term

I’m not sure if this is the best way to solve this problem, but I really like this plugin anyways. split-term.vim is a really cool plugin to improve opening a terminal inside of a NeoVim split. I have vim keybindings set up for my shell, but having those is nothing like having your whole terminal inside of a vim buffer. In fact it’s so good that I doubt I will spend much time inside the actual shell.

Being able to use / to find historical entries and copy/paste things using real vim keybindings is a huge win. Not to mention that I no longer have to ^Z to get back to a terminal, I can just move across splits.

Anyways, the point here is to leverage this plugin to allow us to open the pytest-watch process inside of a terminal generated by split-term so that we can escape the running process and do other stuff while this runs. Fortunately vim-test supports the idea of running “strategies”. There are several built in strategies, but we’ll need to define a custom one for this to work. Going back to our init.vim

" Contents of init.vim

function! TermStrategy(cmd)
    execute ":Term " . a:cmd

    " I don't like to have the terminal be the focused split after being created,
    " so I feed these keys to revert focus to the previously focused split.
    call feedkeys("\<esc>\<C-w>j")
endfunction

let g:test#custom_strategies = {'term': function('TermStrategy')}
let test#strategy = 'term'

I have extremely shallow vimscript experience, so this is sort of hacked together. I’m guessing there is a cleaner way to do this but the snippet is short enough that I’m not concerned for now. Typing <leader>tf again will bring up the split and run ptw then restore focus to the previous split.

Showing code coverage

Compared to the above, showing code coverage is a breeze. Install coveragepy.vim and you can get this working right out of the box.

I added some key bindings to make this a little easier

nnoremap <leader>rc :Coveragepy refresh<cr>
nnoremap <leader>sc :Coveragepy show<cr>

Most of the time I want to keep code coverage hidden because it’s straightforward to keep track of what cases need to be covered and what is already covered by looking at the test names:

def test_some_functionality():
    assert SomeClass().value = 5


def test_a_case_I_havent_covered_yet():
    pass

And then I can show and refresh coverage when I’m finishing up and I want to make sure all the cases are covered.

From here we just have to generate code coverage using the usual command for pytest

let test#python#pytest#options = '-vv --cov module_to_cover --cov-report html'

Full contents of init.vim

" fzf plugin
call plug#begin('~/.vim/plugged')
Plug 'janko-m/vim-test'
Plug 'vimlab/split-term.vim'
Plug 'alfredodeza/coveragepy.vim'
call plug#end()

nnoremap <leader>tf :TestFile<cr>

nnoremap <leader>rc :Coveragepy refresh<cr>
nnoremap <leader>sc :Coveragepy show<cr>

let test#python#pytest#executable = 'ptw --'
let test#python#pytest#options = '-vv --cov module_to_cover --cov-report html'

function! TermStrategy(cmd)
    execute ":Term " . a:cmd
    call feedkeys("<esc><C-w>j")
endfunction

let g:test#custom_strategies = {'term': function('TermStrategy')}
let test#strategy = 'term'

Final result

asciicast

Issues I haven’t solved yet

Ideally I’d like to have the split for ptw always open as the top or bottom split of the editor. I’m sure there’s a way to do this, but I haven’t been able to find it yet.

Additionally, for pytest-coverage, you need to specify the module that you’re currently working with so coverage can be generated. I’m doing that now by having several lines in init.vim that I comment and un-comment as I switch projects which is annoying. There is most likely a better solution for this one as well.

" init.vim

let test#python#pytest#options = '-vv --cov module1 --cov-report-html'

" Comment out inactive modules in other projects
" let test#python#pytest#options = '-vv --cov module2 --cov-report-html'
" let test#python#pytest#options = '-vv --cov module3 --cov-report-html'

Finally, it would be nice to have the code coverage update automatically after each run of the unit tests and show in NeoVim. I don’t mind not having this because I typically only care about code coverage when I’m finishing up writing the tests for a feature and I want to see what types of cases are covered so far. The loop of updating code coverage, writing a new test case, and updating again usually only takes a few passes. It’s not a huge deal to lack this functionality but it’s something I’ll probably keep working on.