computer animation lecture notes

CM50245
Week 19
Computer Animation and Games II
It is an extension of the previous unit.

What We Have Done
Computer animation and game are closely related. Of course there are differences. Games are
about real-time running. Animation - you can do offline work. For animation - everything is
pre-design. For games - you need artificial intelligence to control characters in real-time.

Computer Graphics Sub-areas
Related Graphics Techniques
Course Plan
Last semester we spoke about skeleton deformation. How deformation of the skeleton affects
the deformation of the figure. This semester we are going to talk about non-skeleton
deformation.

Lecture 1
Geometric Modelling
One the four sub-areas of computer graphics.
How to represent shapes, and how to acquire shapes from physical nature, how to analyse
them.

Where is Modelling?
Once we acquired digital model through 3D scanner, we can process this model using various
techniques. Rendering studies how to specify a particular image from a digital figure, how to
apply different materials etc.
Imaging is a counter-product of modelling. It cares about how to capture 2D digital images.
And how to further process 2D images through different applications.
Computer visions - how to reconstruct 3D shapes from 2D figures.

Modelling

Shape Acquisition
3D scanner. Scanning geometry and constructing the shape.
Hand-made shapes. Also a popular way.
These two techniques are the most popular. Other ways are - medical use, CT and MRI
technologies to capture geometry of human organ. Seismic exploration - reconstruction of
geometry under the Earth. But the two first are the most popular.

3D Scanning is Easy Now
Was very expensive before, but now is very cheap. The quality isn’t as great as laser scanner,
but still OK. Microsoft has a Kinet device. You can put it on your tablet and use the app to do
3D scanning now.
KinetFusion: developed by Microsoft research in Cambridge. Gives you an opportunity to
create 3D objects in real time.
[clip]
Digital Geometry
The King Kong character is scanned and reconstructed from the sculpture. First there was a
sculpture and then it was scanned and processed into the animation.
Medical applications use a lot of digital geometry. For new doctors its best to first work on
digital models than on real ones.
3D city modelling is another scenario. City planning.

Shape Representation
Parametric representation. Representation of the shape using two parameters. The sphere longitude and latitude to represent the sphere.
Implicit representation. Using single function to represent the shape.
Mesh representation. Mathematically is very simple. Mesh consists of set of vertexes and a
set of faces.
Different representations have different advantages and disadvantages.
For parametric - we can easily iterate all the points on the surface.
For implicit - we can easily check whether a point is inside or outside of the surface.
For mesh - the storage is simple. It is easy to render. You just need to render all the faces.
Mesh can easily represent shapes with different topologies.

Processing

3D Geometry Processing
A lot of techniques in Photoshop are reproduced in model processing.
Literature
“Polygon Mesh Processing”. In this unit we will cover Reconstruction, Simplification and
Editing.

Geometry Processing Pipeline
1. How to acquire the digital model. The output is raw 3D data. The quality of the data is not
so good.
2. Surface reconstruction. Based on the raw data you reconstruct the mesh model of the good
quality from the raw data. Mesh contains more geometric information than raw data. It has
connectivity between the points.
3. Analysis of surface quality. For instance, the noise level of the reconstructed mesh. Which
part is smooth, which has nods?
4. Surface smoothing for noise removal. Improving the geometry of the model.
5. Parameterisation. Once we have a mesh of the reasonable quality we can performed
parameterisation. To map the 3D shapes onto a 2D plan. If you have this mapping it can help
you a lot if you want to assign textures to the 3D surface. To have a particular colour or
surface effects you need to map 3D to 2D to assign it.
6. Simplification for complexity reduction. You need to reduce the resolution of the input
mesh. So that it can help you to efficiently render the object for computer game.
7. Remeshing for improving mesh quality. Resample the input mesh and have a better
connectivity. If the quality of triangulation is not good. The area of triangles is not evenly
distributed. For some applications this type of mesh isn’t good. You want to have a mesh
with uniform size of triangles. Here you perform remeshing into a higher quality mesh.
8. Freeform and multiresolution modelling. If you want to stretch the model, for example.
Mesh editing will be covered in week 2.

[break]
Lecture 2
3D Data Acquisition
Surface Reconstruction
Digital Michelangelo Project

After scanning the team had to develop software to do the reconstruction to get the complete
mesh model. 1GB sample point from scanning. 8M triangles.
3D scanning techniques
Can be classified hierarchically according to the scheme.

Contact Scanners
Contact between the scanner and the scanned surface. You need to be sure that the scanner is
in touch with the surface. It is a mechanical arm. This is similar to the skeleton structure.
The principle of contact scanner is based on the forward kinematics. It contains angle sensors
which detect movement of angling bones.
The problem - it is very slow. The user needs to place the needle on every surface. So it is not
very popular.
The ant example. The artist first manually draws the mesh on the model. Then the designer
manually places the prone of the scanner at each vertex. Based on this information he can
finally reconstruct the digital 3D shape. This process is very time consuming. He needs to
predeterming the locations he wants to scan. Then he needs to move the probe at each of
these locations. Only then the shape is acquired.
For non-contact scanners the process is much easier.

Time-of-Flight Scanners
First the scanner amids some light to the object. The light hits the object and gets reflected.
The scanner also has a sensor which can receive the reflected light. It can also record the time
difference between releasing and receiving the light. We know the speed of the light, divided
by 2 to compute the distance. So if you amid light at different points of the object you get
different shapes and depth.
Laser LIDAR. The scanning range of LIDAR is much bigger than Kinet. For LIDAR it is
more than 100 meters, for Kinet is 2-3 meters max. LIDAR is good to scan outdoor things.
The accuracy is also better. But the underlying principle is the same.

Triangulation Principle
Passive Stereo or Active Stereo

Passive Stereo Matching
You need to establish correspondence between two cameras. For this you need to establish
some features. Because the correspondence is established afterwards it is called passive,
because it is not the scanner which does the correspondence, but you - afterwards.

We don’t need to project light or laser.
Active Stereo Matching
We need to project some kind of light to the object.
Structured Light Scanner
You need to have a structured, usually black and white pattern, light and the camera which
captures the projection. The camera captures pattern distortion. The key technology is that it
can identify 3D position of the configuration together with the distorted pattern. It can then
reconstruct the information of the captured object.
[demo]
Time-Coded Light Patterns
We need to project black and white pattern several times on the same object. We increase
frequency. Every time it is doubled. The reason for doing this is to identify a point with a
certain resolution. Lets look at the red dot. For the first pattern you can see that it lies in the
black region. But also a lot of neighbouring points. For the next round it is lit by white
colour. Then it gets black again, then white etc. We can see a colour sequence. If we lay the
red dot in another position we get another colour sequence.
Problems: this methods isn’t particularly good in working with textured surfaces. It is also
quite slow because the scanner needs to compute all the patterns constantly.

Laser Scanning
For each sweeping steps the laser produces a fan. The camera computes the fan. Based on the
triangulation principle we know the line. If we use the laser fan to sweep the object you’ll get
many points to compute.
Minolta and Cyberware are examples of laser scanner. Minolta is about 5 kg heavy. So it is
not easy to use on the tripod, for instance.
It is very accurate but rather slow because of the sweeping approach. It needs to sweep the
object from top to the bottom. It also doesn’t work well with reflective and diffusive surfaces.
With reflective surfaces the light will miss the camera. With diffusive - it will be dissolved
before it gets to the camera.

Triangulation Problems
You need to move camera around to avoid the occlusions. One scan is not enough.

[break]

Lecture 3

3D Registration
3D Model Acquisition Pipeline
3D Scanner. We can get the point-cloud from this view point. One scan is not enough. You
need to take several scans from different views. We need to change the viewpoint and scan
another frame. Now we have two scans. But if we are using a single scanner these two scans
are in the same coordinate system. So we need to merge them into the one system. This is
called 3D registration. Alignment. It is based on strong assumption that these two scans have
a lot of overlapping regions. Meaning that some regions are scanned by both scans. Next step
is merging. We combine two scans into one scan. Then we can check again whether we have
the complete scan. If not - we can repeat the procedure until we have a complete shape.

Aligning 3D Data
Red curve is one scan and green curve is another. We can fix the red scan and perform some
rotation and translation to the green one. How to do it? Because we know it there must be
some overlapping regions. Suppose all these correspondencies are even, then it is easy to
solve. We just need to formulate this as an optimisation problem. In practice often we don’t
know the correspondences. The possible way to do this in this situation is to compute some
unique features on the surface. To introduce some features to establish such correspondences.
Another way to do it is based on simple heuristics. We use closest points as correspondence
points. To avoid flows we need a very good initial position. If it is not good - it will not
convert.

Basic ICP Algorithm
Simple heuristics will give you a lot of wrong correspondences. Therefore you need to do
several heuristics. The single iteration will not align the two curves very well, so we need
multiple iterations.

Point Selection
All points will be too complex. The simplest way to do is to perform a uniform or random
sampling. But this sampling ignores the geometry of the whole line patch. The best way to do
is to involve geometry analysis. Which is to distribute points according to the normal
distribution. Stable distribution will give us a better performance.
Closest Point Search
Brute force is the simplest way. But it involves a lot of complexity. N is a number or points.
If they are a lot it will be very slow.
BSP tree or other partitioning is used instead.

BSP Closest Point

With the example, we partition the space into two subspaces. Then we keep on dividing
sub-spaces. ABC stores the splitting plan at different levels. Suppose we are given the red
point and we want to find the closest point within the other points. Within the BSP tree first
we check the relative location of the red point: A. We need to substitution the coordinates of
the point to the creation of the plan. Positive - on the left, negative - on the right. We ignore
the irrelevant regions and keep on traversing the relevant areas. Our point is in region F so we
discard other regions.
Rejection of “Bad” Pairs
There are several rules.
You can use colour as an additional property to charge the alignment.
Find rotation R and translation t to minimise
This formulation if very easy to understand.
Shape Matching: Translation
Idea of solving:
f(r,t)
pf(r,t)It=0
In objective function we don’t have t anymore, we have r.
Shape Matching: Rotation
Shape Matching: Quaternions
A very complicated way of solving.
Multiview Registration
What if you have more than two patches? The simplest way is the one we’ve discussed
before. Suppose we have 6 scans. First we align and merge first two scans, then we add the
third scan to the ones which are already merged. Then we add the forth to the merge of first
three. Finally we align the last one with the previous ones. But it has a problem with the
initial condition. If it is not good the alignment will be affected. It will have some errors. So
for each step you introduce some error which is propagating along the sequence. When you
align the last patch you find out that it cannot be aligned perfectly.
The way to avoid it.
The simplest way is to simultaneously transform all the patches instead of transforming one.
In this case for each of the scan you have rotation and translation to solve.

Multi-Match Registration
You don’t do the merging too early. It helps us to avoid the error.