PyLint

Before you get all excited, PyLint is not finished. However, it does do useful things, so try it out. [ download ]

Objective

• allow modules to be annotated with fixed static type information
• determine the types and interfaces which appear at any type-site in your program, through whole program analysis
• output this information into a ctags like format for use in creating type-aware Intellisense-like completion.
• store inferred-type information across multiple runs of pylint as your program changes
• Generate simple "likely typing errors" based on the inference graph AND comparisons with previously stored infered type information.

We are not concerned with using this information to do any type of compiling or performance improvement.

Background

There are a few different tools available which check Python programs for common errors. Pychecker is very similar to a C-lint tool, reporting bad practices in Python code. Logilab's pylint (not to be confused with the pylint you are reading about right now) checks python code to see if it meets a set of coding standards. Our type-inference based Pylint attempts to go a step further and report programmer typing errors which will fail at runtime. Because it is an analysis tool, it is completely optional, and any items it reports can be ignore.

In addition to performing static analysis, there are other reasons to use type inference to determine type information from Python programs. When reading another programmers dynamic-typed code, it is often difficult to understand what type of object a particular variable might hold. This is particularly tricky when many different types of objects are passed around.

Languages with static typechecking solve this problem by burdening the programmer with additional work while they are writing the application. While some of this work might be useful in disambuigating confusing type usage, much of it is just 'crossing I-s and dotting T-s' to make the compiler happy. PyLint offers a different method for solving the same problem. By performing analysis of modules and building up persistant type structures, Pylint can provide the programmer type information about Python code. This information might be read from auto-generated documentation, or integrated directly into IDEs or editors.

The most widely know type-inference system is the one used in the programming language ML. ML offers amazing performance, and largly type-declaration-less programming, however, it is not without drawbacks. The rigidness of the type system makes polymorphic interface use problematic, resulting in constructs such as print_string and print_int. Furthermore, the type-inference errors produced often leave programmers baffled, since the compiler does not make a decision about which part of your program is most likely wrong.

Pylint aims to support polymorphic interfaces, and provide simple "runtime backtrace" style errors, which are generally richer than compiler errors, but easy to read. While this seems like a much more complex problem than ML type-inference, especially given that Python was not designed for inferred types, Pylint is not overly complex. By relying heavily on annotated type descriptions for library modules, and by storing Pylint created type descriptions for your program, Pylint simplifies the work it has to do in any single invocation. Furthermore, we accept that PyLint will not work in all cases, and we consider it acceptable for Pylint to either require you provide type annotations, or lose type-checking capability, in cases which are too complicated for it to infer.

Overview

Before continuing, we need to define some terms which will be used:

• Type: Number, String, Float, or a particular type of object
• TypeSite: a location where a type can occur, such as a local or global variable, function argument, return value, etc.

Stage 1, Parsing

Stage 2, TypeSite Description

   def a(an_arg):
     print an_arg

   a(1)                # pass in a number
   a("name")           # pass in a string
   a(open("foo","w"))  # call the builtin open and pass in a File object

     Any <- a(int)
     Any <- a(string)
     Any <- a(Obj<__builtin__.File>)

1. It can be diffed against a previous listing to check changes in type signatures
2. it can be loaded into an editor in a ctags like manner to allow programmers to ask the editor for the "possible types" of an argument or variable, and thus "look up" the implementation of that type.

Stage 3, Inference Type-Check

In addition, a syntax for describing static protocol-conformance or type-conformance can easily be developed which this tool will understand and check. For example, this directive:

   def a(an_arg):
     "PYLINT: None <- (int)"
     print an_arg

   def a:None(an_arg:int)
     print an_arg

In the current implementation, type annotations are supplied by way of an example. That example may be fully within a docstring, or in the case of complex cases, in the form of a code example. Below you can see the file which initializes the type annotations for the Python file open() and standard file interface. Notice that basic types are provided directly in the example, however, the open() implementation instantiates the __PythoneFile__ object and returns it.

def int(x):
  'PYLINT (0,), "x", (1.0,1,"",1L)'

def abs(x):
  'PYLINT (1.0, 1, 1L), "x", (1.0,1,1L)'

class __PythonFile__:
  def read(self, n):
    'PYLINT ("",),"n",(1,)'
  def write(self, s):
    'PYLINT (0,),"s",("",)'
  def close(self):
    'PYLINT (None,),'
  def fileno(self):
    'PYLINT (0,),'
  def readlines(self, n):
    'PYLINT ([""],),"n",(1,)'
  def readline(self):
    'PYLINT ("",),'

def open(fn):
  'PYLINT None,"fn",("",)'
  fp = __PythonFile__()
  return fp

  a = 1

import test14

c15 = test14.C()
a15 = c15.m1()

class C:
  def m1(self, a=5):
    "test of default parameters"
    return a

c = C()
aa = c.m1()
bb = c.m1(5)

import test14

c15 = test14.C()
a15 = c15.m1()

b15 = c15.does_not_exist()