Writing Jython Scripts

Data can be processed or accessed in the Python window of FireCode.   The internal data is in the form of Java objects, which might have been created by either Java source code or Scala source code.   In either case, the fields of an object are generally extracted as obj.field() and the methods are used as obj.method( input1, input2, ...).  

In the text below, the notation [x] means an array of objects of type x.   A function call foo( x, y, z="abc") means that arguments x and y must be given, but z can be skipped and a default value of "abc" will be used instead.

The Java objects of FireCode can be accessed directly in Python.  The most common objects (workspace, experiment, plate, FCS record, analysis object) have Python wrappers to simplify access.   

• FireCodeWorkspace <-> PyWorkspace
• Experiment <-> PyExperiment
• FCS <-> PyFCS
• FireflyAnalysis <-> PyParticles
• ExpData (a plate) <-> PyPlate

Pointers to examples are given at the end. 

A PyExperiment hides any details of how the the particles were calculated.   It shows the same data whether it came from the original FCS file or from a saved FWS file.   To get more detailed particle calling and decoding data, use a PyParticles object.

The wrapper objects will be defined shortly.   The following functions generate them.

• loadworkspace( filename).   Read a FWS file and return the corresponding PyWorkspace
• getFCS( filename).   Read a filename and returns the list of FCS or FCI objects as [PyFCS] with no processing.
• loadImages( folder, output).  Read the images in a folder and process into a pseudo-FCS file output.fci.  Returns a list of pseudo-FCS records, one per well of the plate scanned.
  
• analyzeFCS( fcss, plx).   Take a list of PyFcs records and a PLX filename and process the data, returning an array of PyParticles objects.    If you don't have a PLX file or you don't care about decoding, you can provide FireflyAnalysis.default() instead.  If you know the barcode,  vget(FireflyAnalysis.barCode2PLX("229130")) will do the job.
  
• fcs2experiment( fcss, plx, name).   Take a set of FCS records and a PLX file, process into a PyExperiment called name.

For example, to proceed from raw images to a decoded experiment

fcss=loadImages( "myfolder/tiffs", "output")
e=fcs2experiment( fcss, "panel.plx", "Experiment1")

Object PyExperiment defines the samples and probes and data in an experiment, and is the most common way of dealing with data unless detailed decoding information is needed.   It is created either using PyExperiment( e) where e is an existing Java Experiment, or using the fcs2experiment function.   It has the fields and methods

• .e - a pointer to the Java experiment
• .samples - the list of samples
• .probes - the list of probes
• .ns and .np - the number of samples, probes
• .rawdp( s, p) - the raw data for a sample and probe
• .outdp( s, p) - the processed data for a sample and probe (what is shown in the bargraphs)
• .variables() - returns the list of variable names
• .variable(name) - returns the values of a variable, one for each sample
• .probe(name) - find a probe using an abbreviated name
• .sample(name) - find a sample using an abbreviated name
• .wells( pattern) - find the samples matching a regular expression, e.g. wells("[AB].*")
• .probesigs( name, raw=True) - return an array of the signals for a probe, either raw or processed
• .negatives( pattern) - returns a new experiment with the wells matching the pattern as negative controls.   The original experiment is not modified.
• .dilutions( pattern, ratio=3) - returns a new experiment with the wells matching the pattern as a dilution series
• .copy( name, data, samples, probes, dataframe, eopts) - make a new experiment by copying an old one but changing one or more inputs.
• .withData( data3) - make an experiment with synthetic data in an array of array of floating arrays   The outer index is the samples, the middle index is the probes and the inner index is the values for each particle.
• .toFWS() - generates a workspace from an experiment.   Mainly useful for saving.

•     .fws  - the Java workspace, which has the method for storing to file
          .save( string)
•     .plates - the list of PyPlates (can be empty)
•     .experiments - the list of PyExperiments
•     .experimentByName - a dictionary to help finding an experiment by name

• .fcs is a pointer to the original Java object
• .name is the record name
• .parms is a dictionary of parameters
• .colnames are the names of the columns
• .n is the number of events
• .events is an array of events, where each event is a floating array with one entry per column

• .ffa - pointer to the original FireflyAnalysis object
• .name is the name of the FCS or FCI record
• .nevents is the number of events
• .colornames is the list of colors, usually ["green","yellow","red","red2"]
• .colors is an array of arrays, one for each color
• .fcstime is the time array if the cytometer recorded it, otherwise just a list of zeroes
• .logcolor(i) returns the log of color number i for i=0..3
• .particles is the array of particles before decoding.    Each particle has
      .ix the three indices of the FCS events in the order they were found
      .i is the probe index
      .il is the leading index
      .it is the trailing index
      .i1 is code1 and .i2 is code2
      .signal() is the red value of the probe
      .code1() is the floating value of code1
      .code2() is the floating value of code2
• .nparticles is the number of particles
• .codes is an array of code indices, one for each code
• .ncodes is the number of codes
• .codeparticles are the particles for each code
• .codenp are the number of particles for each code
• .codenames are the names of each code
• .codex and .codey are the coordinates of the middle of the ellipse
• .codea, .codeb and .codecq are the long and short axes of the code ellipse, and its rotation
• .pnames are the probes
• .pstats are the stats for each probe

Plotting

• plot( seq, w=400, h=400)
  The main entry point.  Any sequence of plot objects is assembled and rendered in a window of size w x h.   A couple of special cases are supported.   If seq is a list of floating point numbers instead of a list of plot objects, it generates a scatter plot of the values in order.   If there are two sequences, it plots the second against the first.
• plotpanel( seq, w=400, h=400)
  Like plot, but instead of drawing it on the screen, returns it as an object which can be saved in Excel
• colorlist(n)
  Utility for creating a list of colors of length n
  
• axes( "x", "y","t") - returns an array of three plot objects, one for the X axis, one for the Y axis and one for the title.   There can be an extra argument log which is a string containing "x" if the x axis is log and "y" if the y axis is log.
• legend( ts, cs)
  Give a list of titles and a list of colors
• textbox( textseq)
  A floating box of text, like information about a regression fit
• xstretch( xlo, xhi) and ystretch( ylo, yhi)
  When added to a plot sequence, reserves at least the range xlo to xhi or ylo to yhi for X and Y.   Useful if there are many plots and you want them all to have the same range even if their data is different.
  
• scatter( xs, ys, color=Color.blue, size=6.0, symbol="", log="")
  Make a scatter plot of the ys against the xs.   The symbols are as follows.   Adding a period makes a symbol solid.
  "o" circle
  "[]" square
  "<>" diamond
  "^" "v" "<" ">" triangles up down left right
• lines( xs, ys), color=Color.blue, size=6.0, symbol="", log="")
  Like scatter but with a line.   Size=0.0 suppressed the symbols.
• barplot( xlab, ylab, title, categories, seqseq, gp=None, render=True, w=400, h=400)
  Unlike the functions above, barplot assembles an entire plot and shows it.  If you want to capture the plot sequence without showing it (to save in Excel) use
  foo = barplot( ..., render=False). 
  The first three arguments are the X, Y and title strings.   Categories is a list of strings to put under each bar.   SeqSeq in array of arrays.  There is an array of values at each bar which is represented by the inner array.   For a plain bargraph with just one value at each point, use
  [ [d] for d in data] to convert each individual value d into an array of length 1.
  The parameter gp is a GraphParam object and controls log scale, appearance of individual dots, etc.  It is most easily created like
  from com.firefly.rep import GraphParam
  GraphParam.default().setLog()
  Other options are .setShowPoints(), .setShowCounts(), .setBoxWhisker(), .setYmin(y) .setYmax( y)
• multibarplot( xlab, ylab, title, cats, subcats, seqseqseq, gp, render, w, h)
  Similar to barplot, but as well as categories, give subcategories and the array of arrays becomes an array of arrays of arrays.   E.g.
  multibarplot( "sample","signal","pa vs pb",["s1","s2","s3"],["probe a","probe b"],
  [   [   [1],[2] ],  [ [10],[20] ],  [[30],[40]] ] )
• histogram( vec, breaks)
  Makes a bargraph from the values in vec with breaks from the given array
• distogram( vec, name="x", resolution=0.02, render=True)
  Like a histogram but smoother.   The smoothing defaults to 2% of the range of x values
• distograms( vecs, resolution=0.02)
  Multiple superimposed distograms
• cdfplots( vecs, name)
  Shows the cumulative distribution function of multiple vectors superimposed.   For just one vector use [vec].
  

GUI programming

Interactive GUI functions can be built using a set of wrappers above the underlying components from Java's Swing library, including JButton, JCheckbox, JLabel, JFrame, JPanel, JTextBox.   A full discussion of Swing is beyond the scope of this document, but in a nutshell there are components, which are laid out in panels, and the panels are shown in a frame.   The wrapper functions defined by FireCode are

• askuser( question, panel)
  Gives a user a yes or no choice.   For instance
  if( askuser("Do you want to proceed", parent_panel)):
     do_something
  else:
     println("Not proceeding")
  
• telluser( msg, panel)
  Like alert(msg) but less alarming
• showuser( obj, msg, ppanel):
  Show a user an object and ask whether to continue, e.g.
  if( showuser( obj, "Options", parent_panel)):
      do_something
  Very commonly the obj will be another GUI with various options the user can change, and do_something reacts according to the values of those options.   See the function run_scripts() in the file 6scripts.py of the standard library for an example.
• filechooser( ftype, fext, multi=True, save=False, dir=False, sugg=None)
  Open a file chooser.   ftype is text describing the kind of files being opened.  fext is a list of accepted extensions.   Multi is whether we accept multiple files.  Save means writing a file instead of reading a file.  dir means we are looking for a folder.   sugg is a suggested name
• jshow( comp)
  Pop up any component in a separate frame
• vbox( comps, margin=[2,2,2,2], line=0) or hbox
  Create a panel with a list of objects in a  column (vbox) or a row (hbox).
  Margin is the amount of space between the array and the edge of the panel (T,L,B,R)
  Line is an option linewidth drawn around the panel
• tablep (colHeads, data, w=400, h=600)
  Create a scrollpane containing a table with the headers defined by colHeads and array of arrays, with one array for each column and the length of the arrays being the same.   Swing will do its best to present different types of array appropriately, e.g. strings, integers, floating point.
  

Excel reports

PyXLSX is an Excel wrapper  built using python dictionaries.  Each sheet has a dictionary with entries for A01, A02, etc.   There is also a style for each cell.   Cell A01 in sheet Sheet1 in the spreadsheet object xl is referenced as xl.dicts["Sheet1"]["A01"], and its style is xl.styles["Sheet1"]["A01"].
The following functions are defined:

• creation: xl=PyXLSX()
• xl.getSheet("S1")
  Creates a sheet called S1
• xl.setcellref( "S1","A01",value)
  Puts the value into cell A01 of sheet S1.   Can also be written xl.dicts["S1"]["A01"]=value
• xl.setcellrc( "S1", 4, 2, value)
  Puts the value into the 4th row, 2nd column of sheet S1.  The 0,0 cell is A01.
  
• xl.encode(r,c)
  Returns the corresponding cell to row r and column c
• xl.decode("A01")
  Returns the row and column corresponding to the string, in this case (0,0)
• xl.getcell("S1","A01")
  returns the value of the cell
• xl.scalePlot( plotseq, cols, rows, sheet)
  Inserts a plot into a sheet.   The plotseq is a list of plot objects, as would be passed into the plot() command for display.   Cols is the lowest and highest column used for the picture, and rows is similar.   Plots generally scale best if the number of columns is about 4X the number of rows.
• xl.insTable( headers, data, startcell, sheet, color=None, bstyle=None)
  Inserts a table into a sheet.   Headers is a list of strings.  Data is organized by column.  Startcell is a reference like "A01".  Sheet is a string.   color is a background color for the table.  bstyle is a tuple (linestyle, Color)
• xl.dict2sheet( sheet)
  Transfers the data from dictionaries into an Excel memory object.   Not usually necessary, since save() calls it.
• xl.defaultstyle()
  Returns the default style of the spreadsheet
• xl.updstyleref( sheet, ref, func)
  Updates the style of cell ref in the sheet, using function func that modifies a style.
• xl.updstylerc( sheet, r, c, func)
  Like updstyleref, but uses the row, column coordinates.
  
• xl.save( fname)
  Saves the Excel memory object to file called fname.

• setFill( color)
• setBorder( side, linestyle, color)
  Defines a border of a cell.  Side is one of "top","bottom","left","right".  Linestyle is one of "none", "hair", "thin", "medium", "thick", "---", "...", "===", "-.-", "_..","m-.-", "m---","m-..", "/-." . Color is a java Color (Color.blue, Color.red, Color(255,128,0), Color(1.0,0.5,0.0), etc)
• setHalign("left" or "center" or "right")
• setValign("top" or "middle" or "bottom")

Example: xl.updstyleref("S1","A01", lambda x: x.setBorder("right","m--",Color.blue))

Built-in variables

These variables are built into the interpreter

• none.   The scala value None.   Python has its own value None, which is not the same.   In Scala, an Option[Something] can hold either a something as Some(value) or None.   There are some FireCode functions that take an input of the form Option[Sample] or Option[Probe] and if no input is desired, the value "none" can be passed, otherwise Some(sample1) would be used.
  
• NaN.   The double precision value that represents not a number.   Useful for when a computation cannot be completed, e.g. sqrt(-1).   Note that if(x!=NaN) is not a useful test of see if x is a good number, because by convention nothing is equal to NaN, including NaN itself.  Use the built-in function isnum(x).
  
• plotter.   The plotting object, which has a variety of methods defined on it.  Generally these methods are best accessed through the plot library, but one method that is useful is plotter.clearAll() which removes all the popup graphs that have been generated.
• console.   Exists to provide the function console.log so that output can be written to the screen.  The input must be converted to a string before printing, e.g.
  console.log( str( [1,2,3] ) ).
• workspace.   The workspace from FireCode, as a raw FireCodeWorkspace object.   Generally it is more useful to use PyWorkspace( workspace).   The workspace is updated when the interpreter is started, and every time the "update workspace" button is pushed.
• parent_panel
  Some GUI functions need a link back to the Firecode frame; parent_panel is the link.
  

Built-in functions

These functions are built into the interpreter, or are part of the standard library.

Input/Output

• showlist( xs, format=None).    Print one line per object in the list.   If a format is given, like "%d" or "%8.5f" for integer and floating point numbers, it will be applied to the members of the list.
• println( x).   Just a shorthand for console.log( str(x))
• alert(x) pops up a window with the message x.    Use sparingly; if it's inside a loop it could create a lot of windows which you have to kill with the mouse one by one.
• csv( xss, file, fmt="%.2g") save an array of floating arrays to a csv file, using 2 decimal points
• wellname( string) extracts the well name from the middle of some long string
  

Sequences

• fromseq( xs).   Take a Java or Scala list/vector/array and make it into a Python array

• toseq( xs).   Take a Python array and make it into a Scala List.   
  
• rep(x,n) returns an array with n copies of x
• flatten( lists) turns a list of lists into one long list
• transpose( array) returns a new array with the rows and columns flipped
  
• zzip( xs, ys) is a safer version of Python's zip that checks the two lists are the same length before zipping them together.  
  
• unzip( zipped) is the inverse of Python zip.  unzip( [ [1,4], [2,5], [3,6] ]) = [ [1,2,3], [4,5,6] ]
  
• groupBy( array, catfun):  Organize an array by a categorization function, e.g.
  groupBy( [1,2,3,4], lambda x: x%2) gives a dictionary {0: [2,4], 1: [1,3] }
• count( xs, func) returns the number of elements of xs that func(x) is true on, e.g.
  count( somelist, lambda x: x>0)
• exists( xs, func) returns true if func(x) is true for any x in xs
• forall( xs, func) returns true if func(x) is true for every x in xs
  

Numerical functions

• isnum(x) checks to see if x is a valid number.  
  
• tofloat( x) converts anything to a floating value, NaN if it doesn't make sense
  
• frange( start, end, n) generates an array of n floating values between start and end
• ratio(ys,xs) returns an array with each element of ys divided by the corresponding element of xs
  
• add(xs,ys) returns an array with eacy y added to each x
• mul(xs,ys) returns an array with eacy y multiplied by each x
• neg(xs) returns an array whith the negative of each x
• mean(xs) median(xs) geomean( xs) different averages of the array xs.  Bad values are skipped
  
• sd(xs) standard deviation of array xs
• CoV( xs) coefficient of variation of array xs (sd / mean)
• cov( xs, ys) covariance of xs and ys
• corr( xs, ys) correlation coefficient of xs and ys
• reg( xs, ys) regression object of ys against xs, containing
  .slope .intercept .r2 .rho
• rms( xs) the root mean square of the array xs
• integ (xs, ys) integrate area under a curve defined by a y at each x
  

Error Handling

In Firecode, many functions that might result in a value but which might fail are returned as a Vali[Object] which can contain either Valid( result) or an error message.

• vget(xv) returns the value of xv if xv is valid, or None if not
• verr(xv) returns the error that caused xv to be invalid

if( vget(xv) != None):
    do something with vget(xv)
else:
    println( "HELP: "+ verr( xv))

Examples

Troubleshooting

See Also