The sensor noise signal can be estimated by averaging imperfect noise estimates from many images. Correlation indicates how strongly the signal appears in a test image.

There is more information on my GitHub page.

An investigator can try to work out what interpolation method was used via a statistical analysis of the image's pixels. This can give an indication of what camera make/model or post-processing software was used to produce a untampered test image.

Areas of the image that are inconsistent with an expected interpolation pattern may have been tampered.

There is more information in my slides on colour filter array interpolation detection.

There is more information in my JPEG tutorial and slides on JPEG compression history analysis.

• Context-Based Adaptive Binary Arithmetic Coding in the H.264/AVC Video Compression Standard (Detlev Marpe, Heiko Schwarz and Thomas Wiegand)

H.264/MPEG-4 AVC defines several different profiles, which specify which coding methods and parameters are allowed in a stream. The Main and High profiles allow the use of context adaptive binary arithmetic coding (CABAC), which offers improved compression performance (around 10% bit saving) compared to the context adaptive variable length coding (CAVLC) method which is available as an alternative, though it is more expensive to implement.

Overview of the entropy coding process

The entropy encoder takes as input a sequence of symbols representing samples and control information and maps this onto a binary bitstream which is output into the container. In contrast with earlier compression stages, the entropy coding is lossless; the decoder will reproduce the exact sequence of symbols which was input to the entropy encoder during compression.

H.264's implementation of CABAC creates the bitstream in three stages.

1. In binarization, each symbol to be output is uniquely mapped onto a binary string, called a bin string. Each bit position in the bin string is called a bin. Each bin is then passed to one of two coding modes: in regular coding mode, the next step, context modelling, is applied and the resulting context model and bin value are passed to the binary arithmetic coding engine; in bypass mode, context modelling is skipped and the bin is passed directly to a bypass coding engine, skipping the context modelling stage.
2. In context modelling (only used for regular coding mode) a bin is categorised for coding under a particular probability model. Each probability model has its state represented by a context variable which is a pair (most probable symbol in {0, 1}, probability of less probable symbol). Arithmetic coding is applied using the chosen context model and updates its context variable.
3. In binary arithmetic coding the value of the bin is used to update the context variable if applicable, and bits are output into the bitstream.

Binarization

The input to this process is a symbol (syntax element) to be coded, such as a quantization transform coefficient, macroblock type specifier or a motion vector component. The mapping onto bin strings should be close to a minimum redundancy code.

Main types of binarization

Four main types of binarization are defined:

• Unary code – the value x ≥ 0 is mapped onto x 1 bits followed by a 0 bit.
• Truncated unary (TU) code – the value 0 ≤ x ≤ S is coded with a unary code if x < S, or x 1 bits otherwise. If the condition 0 ≤ x ≤ S holds, the truncated unary encoding of value x is given by

  def tu(s, x):
    for i in range(0, min(s, x)):
      put(1)
    if x < s:
      put(0)

• kth order Exp-Golomb (EGk) code – the value x is mapped onto three sequential bit strings: a prefix, suffix and sign bit. The construction of a kth order Exp-Golomb code for value x is given by

  def egk(k, x):
    while True:
      if x >= (1 << k):
        put(1) # bit of the prefix
        x = x - (1 << k)
        k = k + 1
      else:
        put(0) # end of the prefix
        while k > 0:
          k = k - 1
          put((x >> k) & 0x01) # bit of the suffix
        break

• Fixed-length (FL) code – the value x < S is mapped onto its binary representation, using ceil(log₂S) bits.

There are also five unstructed binary trees defined manually for coding macroblock and submacroblock types.

Concatenated binarizations

The codes can also be concatenated. There are three situations where concatenations of the four basic types are used:

• coded_block_pattern is encoded using a 4-bit FL prefix (for luma) and a TU suffix with S = 2 for chroma.
• Motion vector differences are encoded with a concatenation of a unary prefix and a 3rd order Exp-Golomb code suffix: for a value mvd, the prefix is a TU coding with S = 9 of the value min(|mvd|,9), or, if mvd = 0, just the bit 0. If |mvd| ≥ 9, a suffix is output with the value |mvd| - 9 using the EG3 code. A sign bit is then output if |mvd| > 0: 0 if mvd is positive and 1 otherwise. The following code performs this coding, referencing the coding procedures for the main types of binarization above.

  def uegk(s, k, x):
    absx = abs(x)
    tu(s, absx)
    absx = absx - 9
    if absx >= 0:
      egk(k, absx)
    sgn(x)
    
  def sgn(x):
    if x == 0:
      return
    if x < 0:
      put(1)
    else:
      put(0)

• Absolute values of transform coefficient levels (coeff_abs_value_minus1 = abs_level - 1 is coded, as the positions of zero-valued coefficients are specified in a map) are coded using a TU prefix with S = 14 and an EG0 suffix.

Context modelling

The context modelling stage associates a context model with each bin output by the binarization stage.

There are four basic types of context modelling, which associate a probability with each bin based on previously coded values or other symbols in the neighbourhood:

• Up to two neighbouring syntax elements are chosen based on the syntax element being coded, and the context model for the bin being coded is chosen based on the context model of the related bin in the neighbour syntax elements. For example, the context model of the related bin in the syntax elements in the above and left bins may be selected for the current bin.
• For the mb_type and sub_mb_type syntax elements, the model for a bin b_i with prior coded bins (b₀,b₁, …, b_i-1) is chosen based on those prior bin values.
• On residual data only, based on position in the scanning path
• On residual data only, based on number of encoded levels with a particular value prior to the current level bin being coded.

The context modelling process only ever references past values within the same slice.

Each syntax element may use one of a range of models, each of which is denoted by a context index. The possible models for each syntax element are given in Table 9-11 of the standard, which specifies the allowable values for the context index γ for each element. The range of allowed values of context index for mb_type, sub_mb_type and mb_skip_flag depends on the slice type being coded (SI/I, SP/P or B).

Each probability model (uniquely associated with a context index) consists of a pair of two values: a 6-bit probability state index σ_γ and a single bit which is the most probable symbol (MPS). Each model is therefore represented by a 7-bit value.

Macroblock type, submacroblock type, spatial and temporal prediction modes, slice- and macroblock-based control information syntax elements all use context indices between 0 and 72. The context index is calculated as γ = Γ_S + χ_S where Γ_S is the context index offset, which is the lowest value in the allowable range for the syntax element's context index, and χ_S is a context index increment, which specifies the offset within the range. χ_S may either depend only on the bin index (giving a fixed assignment of probability model to each bin), or it may specify one of the first two context modelling types above.

Context indices in the range 73 to 398 are used for coding residual data (except for γ = 276 which is associated with the end of slice flag).

significant_coeff_flag and last_significant_coeff_flag use different models depending on whether they are in frame or field mode. Not all context models are used in frame-only/field-only pictures.

The model for coded_block_pattern is specified using γ = Γ_S + χ_S. All other syntax elements of residual data use the relation γ = Γ_S + Δ_S(ctx_cat) + χ_S, where the context category dependent offset Δ_S. Table 9-40 in the standard specifies the value of this offset, in terms of the context category which is given for each block type in Table 9-42.

Binary arithmetic coding

Arithmetic coding works by representing an interval within [0, 1] by two values: a lower bound L and a range R, and recursively subdividing this interval using the probability and value of each input bit: on reception of a more probably symbol (MPS), with probability p_MPS, the interval is updated to have width R_MPS := R ⋅ p_MPS (the corresponding operation for a less probable symbol would update the interval to have width R_LPS := R ⋅ p_LPS then update the lower bound L := L + R - R_LPS).

H.264/MPEG-4 AVC CABAC uses a modulo-coder (M coder) as its binary arithmetic coding implementation. It avoids the multiplication above by quantizing the value of the interval width R onto a small set of values Q = {Q₀, Q₁, …, Q_{K - 1}}, and the probability range of the the less probable symbol p_LPS (0, 0.5] onto another set of values P = {p₀, p₁, …, p_{N - 1}}. The tradeoff chosen for H.264 was K = 4 quantized range values and N = 64 probability vaulues.

For the bypass-mode coding engine, the probability estimation stage is omitted.

My slides on CFA interpolation detection include a more thorough introduction to the topic.

The information here is based on published work in the area. I prepared some of the code for Markus Kuhn's forensic signal analysis course at the University of Cambridge Computer Laboratory, which I co-lectured in 2009 and 2010.

As part of a literature survey on multimedia forensics, I compiled a multimedia forensics bibliography. The pages are generated using the Django web framework for python, using content from an SQLite database which I populate using a custom python script which parses a BibTeX file. If you would like to present your BibTeX bibliography in a similar way, contact me for source code.

JavaScript variables are either Objects (including functions) or one of the primitive types: Boolean, Number, String, null or undefined.

Objects may have prototypes. Prototypes are objects which themselves may have prototypes, forming the finite-length prototype chain. An object with a null prototype ends the prototype chain (Object.prototype has a null prototype).

When a member is accessed via the dot operator, each prototype in the prototype chain is checked in turn, until the named member is found, or the null prototype is reached, in which case undefined is returned. The internal [[prototype]] member, referring to an object's prototype, is not publicly accessible.

Inheritance and shared members are implemented using prototypes in JavaScript. The non-standard, settable __proto__ property can be assigned an object to cause that object to inherit from the assigned object.

While a particular method is being executed, the this keyword refers to the object on which the dot operator was applied. This means that in an inherited method it still refers to the subclass. this always refers either to (1) the global object (which is window in a browser) outside a function or inside a function invoked via a variable, (2) the owner of a property access, (3) the value of the first argument passed into Function.prototype.call/apply, (4) the newly created object in a constructor, (5) the calling context's this in evaled code.

Function.prototype.bind takes an object and returns a function which, when invoked, will have a this value equal to the object passed to bind.

To create objects with the same structure but different state, we use constructors.

During JavaScript execution, a stack of execution contexts is created. Specifically, global code gets an execution context, and each invocation has an associated execution context. evaled code also has a distinct execution context. When a function returns, the current execution context is popped from the stack. When an execution context is created, the following takes place:

1. A special Activation object is created. This has no prototype, but does have accessible named properties.
2. An arguments object is created. This maps integer indices onto the corresponding actual parameters of the function, and has callee and length properties.
3. The context is assigned a scope chain. Each function object has an internal property, [[scope]], containing a list of objects. The scope for the new execution context consists of the scope chain of the function object under execution with the newly-created Activation object prepended.
4. The Activation object is also a Variable object. This contains properties for each of the function's formal parameters, assigned the the values of the actual parameters (or undefined when not present). Any inner functions create function objects that are kept in this Variable object. Finally, local variables declared in the function are stored in the variable object in properties according to their names. The value of a local variable is only assigned during execution of the relevant line of the function body (taking into account hoisting), but is initially undefined.
5. The this keyword is assigned. If the assigned value is null, property accesses refer to the global object.

The global execution context does not have an arguments property, but its variables object is created in the normal way, including 'local' variables and function definitions, which appear as global variables and top-level functions.

Closures

Statements in inner function bodies may access local variables, parameters and declared inner functions within their outer functions. When it is made accessible outside the function where it is declared, a closure is formed, and it continues to have access to those variables.

This page contains information about my JPEG exact recompressor implementation. If you would like to get the source code, please read the instructions below then click here to download rjpeg-0.8.tar.gz.

rjpeg: Exact JPEG recompressor (version 0.8)

Introduction

In our paper 'Exact JPEG recompression' (Andrew B. Lewis and Markus G. Kuhn) we presented a technique for calculating the JPEG bitstream(s) which produce a particular uncompressed image given as input.

For full details of the algorithm, see our paper: http://www.cl.cam.ac.uk/~abl26/spie10-full.pdf

For an overview of the algorithm, a poster is also available: http://www.cl.cam.ac.uk/~abl26/spie10-poster.pdf

This archive contains the source code for the recompressor implementation described and evaluated in the paper.

Please note that this software is experimental and should not be used in production software. Error checking is missing, some debugging code is included in the source and the code has not been tested/optimized thoroughly.

See LICENSE for licensing information.

If you find this software useful, I would be grateful to receive an email describing how you have used it (andrew.lewis at cl.cam.ac.uk). If you would like to refer to it in an academic publication, please cite our paper:

@conference{lewis:75430V,
  author = {Andrew B. Lewis and Markus G. Kuhn},
  title = {Exact JPEG recompression},
  publisher = {SPIE},
  year = {2010},
  journal = {Visual Information Processing and Communication},
  volume = {7543},
  number = {1},
  eid = {75430V},
  numpages = {9},
  pages = {75430V},
  location = {San Jose, California, USA},
  url = {http://link.aip.org/link/?PSI/7543/75430V/1},
  doi = {10.1117/12.838878}
}

Please send any bug reports, queries, suggestions or patches to andrew.lewis at cl.cam.ac.uk.

Archive contents

• README, LICENSE, Makefile: Makefile has targets for the main application (rjpeg) and a version for use with the Condor distributed computing system (www.cs.wisc.edu/condor)
• rjpeg.h, rjpeg.c: main(...) function
• data.h, data.c: Data types for pixel data, sets, intervals and expression trees
• computations.h, computations.c: Rearranges expression trees for chroma smoothing
• cspace.h, cspace.c: Inverts the colour space conversion
• diagnosticinformation.h, diagnosticinformation.c: Functions to output infeasible block information
• fdctislow.h, fdctislow.c: IJG forward DCT (`slow', integer)
• forwardoperations.h, forwardoperations.c: Searching and filtering of blocks of quantized coefficient intervals
• jpegout.c, jpegout.h: Use libjpeg to output JPEG bitstreams
• quantize.c, quantize.h: Calculate possible quantization matrices and apply quantization
• reverseidct.c, reverseidct.h: Reverse the decompressor IDCT using libgmp arbitrary precision arithmetic
• solver.c, solver.h: Apply the chroma unsmoothing algorithm
• unsmooth.c, unsmooth.h: Generate the expression tress for chroma smoothing

Input requirements

rjpeg takes a file in PPM P6 (binary 24 bits/pixel) format. (Multiple files can be specified and are processed separately.)

rjpeg will run to completion if the input image was output by a process equivalent to applying the IJG djpeg algorithm on an JPEG bitstream with the following characeristics:

• the image was encoded with chroma sub-sampling (4:2:0);
• the stored colour space is YCbCr; and
• the image's width and height are both multiples of sixteen (twice the DCT block size).

Note that these are cjpeg defaults. Also, the decompressor IDCT must be equivalent to the IJG integer 'slow' transform (the default).

If these conditions are not met, rjpeg will output an error message when it encouters an inconsistency. I plan to add more helpful diagnostic information in a later version, to make rjpeg more useful in forensic situations.

Performance

rjpeg stores a 128 MB look-up table for colour space conversion on the disk. By default this is kept in /tmp/ycc_rgb_table but its location can be altered in cspace.h. This is generated whenever that files does not exist, so the program will take longer to execute the first time you run it.

Approximate time/space requirements: 512 by 512 images at qualities 90 and below typically take a few minutes to recompress on my machine, proportional to the quality factor and number of saturated pixels. The maximum memory usage was around 500 MB.

I have not yet tried to optimize speed and memory usage, and there are many opportunities to do so.

How to use

You will need these headers and libraries:

• The IJG library, used to create output bitstreams (libjpeg).
• The GMP arbitrary precision arithmetic library (libgmp).

You may wish to update the following constants:

The filename used to store the colour space conversion table, in cspace.h: #define INVERSE_YCC_RGB_TABLE_FILE_NAME "/tmp/ycc_rgb_table"

The default will produce -rw-rw-r-- 128M /tmp/ycc_rgb_table (An alternative string constant is used in the Condor target.)

The number of quantized DCT coefficient block candidates beyond which the exhaustive search step is considered infeasible, in forwardoperations.hstatic const UINT64 possibilities_limit = (1L << 20);

The upper and lower limits for results of the IDCT, used when inverting the range clipping operation, in reverseidct.h #define RANGE_LIMITING_UPPER_LIMIT 288, #define RANGE_LIMITING_LOWER_LIMIT -35. Making these values further from 0 or 255 will recompress more images with black or white areas correctly at the expense of increased search sizes.

Run 'make' to produce the executable.

I have included a compressed test image from the UCID http://www-staff.lboro.ac.uk/~cogs/datasets/UCID/ucid.html.

Example usage:

$ tar -zxvf rjpeg-0.8.tar.gz
$ make
$ djpeg -outfile example1.ppm example1.jpg
$ ./rjpeg example1.ppm
QF: 40
Y: 3072 exact 0 ambiguous 0 infeasible 0 impossible
Cb: 768 exact 0 ambiguous 0 infeasible 0 impossible
Cr: 768 exact 0 ambiguous 0 infeasible 0 impossible
$ diff example1.ppm.result.jpg example1.jpg

The last command should output nothing, indicating that the files are binary identical.