Advanced Data Sharing Technologies
Content
EDUCATION
Advanced Data Sharing
Technologies – Part 1
version 9
Philippe Nicolas, Symantec
Jonathan Goldick, ONStor
Abstract
EDUCATION
Advanced Data Sharing Technologies – Part 1
The What, Why and How of Data Sharing technologies plus block and filebased approaches
for IT Director and Managers, IT/Storage/System Eng., Admins, Architects
and Trainers
How to deliver more performance and data accessibility with little to no
additional cost? How to take advantage of existing storage infrastructure to
provide more value to end-users and the global enterprise? A clear industry
direction indicates that Data Sharing architectures and technology can be a
good way to achieve these objectives.
The first session offers a definition of Data Sharing and a discussion of its
benefits with examples linked to the SNIA Shared Storage Model. We cover the
main data sharing approaches and describe how they can improve
performance, data accessibility, and manageability. This includes Scale-in and
Scale-out methods based on block, file and application technologies such as:
Cluster Volume Managers, SAN File Systems, Cluster File Systems, Parallel
NFS (pNFS), Object-based Storage Devices (OSD) and Global/Parallel File
System.
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
2
SNIA Legal Notice
EDUCATION
• The material contained in this tutorial is copyrighted by
the SNIA
• Member companies and individuals may use this material
in presentations and literature under the following
conditions:
– Any slide or slides used must be reproduced without modification
– The SNIA must be acknowledged as source of any material used
in the body of any document containing material from these
presentations
• This presentation is a project of the SNIA Education
Committee
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
3
Agenda
EDUCATION
• What is Data Sharing ?
– Definition
• Why Data Sharing ?
– End User Benefits
• How is Data shared ?
– Block and file-based approaches
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
4
EDUCATION
What is Data Sharing ?
Why Data Sharing ?
What is Data Sharing ?
Definition
•
EDUCATION
Shared access to
same data (value & location) by multiple systems
– Read/write: changes to data become visible to all servers
– Read-only access via mechanisms that support shared read/write
access
•
Examples
– Read/write access to a shared file is data sharing
• So is read-only access to a shared file
– Clone/Snapshots (read-only or read/write) are not data sharing
• Changes do not affect original data
•
Caching is data sharing when changes propagate
– Changes to cached data must become visible to all
– Other data changes must become visible via cache
•
Replication/CDP* is not data sharing because location changes
– Potential divergence of data
* CDP: Continuous Data Protection
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
6
Why Data Sharing ?
End User Benefits
EDUCATION
• Better performance and scalability
– Larger server can be expen$$ive
• Sharing: apply more servers to same problem
– Scales well for some applications, poorly for others
– Can avoid replication or cloning
• Concurrency and Content access distribution
– Use same data for more than one application
• Administration
– Consolidated shared resource has lower TCO
– Data Sharing increases the benefits of Storage/Data
Consolidation
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
7
Storage Consolidation
Scale-up by Scale-In
EDUCATION
File
Server
Data Network - LAN
NFS/CIFS
Server
Scale-In
Storage
Network
Shared
Disks
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
8
Performance Improvement
Scale-up by Scale-Out
EDUCATION
Application
Server
Application
…
DB Engine
Cluster Software
Shared Storage Software
Scale-out
Storage
Network
Shared Disks
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
9
Why Data Sharing ?
How to apply Data Sharing to do useful things
EDUCATION
• High Availability Clusters (local & geographic)
• Scaling applications
– Web Servers - Read mostly/load balanced
– Databases/OLTP/DW - Mostly use direct I/O
• Parallel applications and fast failover
• Systems and Applications Consolidation/Migration
• Off-host processing
– Based on shared file system
– Can also use by Point-in-Time copy techniques (not related to
our data sharing definition)
• …address both Performance and Availability with no
administration degradation and overhead
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
10
Some technologies
and products …
IBM AFS
WebNFS
ISO9660
EDUCATION
Apple Xsan
Cisco FileEngine
Coda
VERITAS CFS
RFS
DiskSites FilePort
Distributed,
Samba FineGround
PolyServe Matrix Server
Oracle OPS/RAC
ClusterLUSTRE
or
WebFS
EMC
HighRoad
IBM Storage Tank
IBRIX
FusionFS
SAN
File
System
SGI CXFS
CIFS
DB2
WAFS
ONStor STOR-FS
HP TruCluster/CFS
ADIC StorNext FS
Parallel &Sybase MPP
SUN SAM-FS
Partitioned
Sanbolic Melio FS
IBM SANergy
Applications
SMB
Tacit Networks Ishared
OSD
PVFS
Redhat GFS
PPFS
Sun QFS
Volume
WebNFS
Isilon IQ OneFS
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
OpenAFS
NFS
Informix XPS
DFS
Nuview StorageX
pNFS
11
EDUCATION
How is Data Shared ?
How is data shared ?
Approaches and methods
EDUCATION
• Several levels are possible
– Share at the volume level (block based)
– Share at the file or file system level (file, block or object* based)
– Share at the database or application level (custom)
• In all cases, all these methods could occur
– among like or dissimilar systems (OS),
– concurrently or serially,
– directly at the storage or in the network
* For OSD
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
13
How is data shared ?
Approaches and methods
EDUCATION
• Traditional/Historical
– Block level: Volume Management
– File/File System (FS) level: Local FS (serial data sharing) and
distributed methods with NAS, Samba, AFP (AppleShare), DFS,
AFS/OpenAFS, RFS, Coda…
– App./DB level: custom built-in methods (RDBMS, Email
systems…)
Check
Check out
out
SNIA
SNIA Tutorial:
Tutorial:
• Advanced/Recent - File/FS level
– Distributed: WAFS approach (NAS extension)
and Network File Management/Virtualization
(NFM), Global FS, SANFS and Cluster FS
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
•• Storage
Storage
Virtualization
Virtualization
Check
Check out
out
SNIA
SNIA Tutorial:
Tutorial:
•• NAS
NAS &
& iSCSI
iSCSI
14
How is data shared ?
The SNIA Shared Storage Model
EDUCATION
Application
Application
level
Database
(dbms)
Cluster FS
WAFS
NAS
File system
(FS)
Shared
LVM
SAN FS
Host
Network
Block
aggregation
Device
Storage
Storage devices
devices (disks,
(disks, …)
…)
Block layer
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Discovery, monitoring
monitoring
Discovery,
Resource mgmt,
mgmt, configuration
configuration
Resource
Security, billing
billing
Security,
Redundancy mgmt
mgmt (backup,
(backup, …)
…)
Redundancy
High availability
availability (fail-over,
(fail-over, …)
…)
High
Capacity planning
planning
Capacity
File/record layer
Services
Storage domain
GFS
15
How is data shared ?
Volume level
EDUCATION
LVM
LVM
Data
Path
Data Layout
and
Organization
•
Examples
–
–
–
–
–
–
–
EMC PowerPath Volume Manager (PPVM)
HP Shared Logical Volume Manager (SLVM)
IBM Logical Volume Manager (LVM)
MACROIMPACT SANique Cluster Volume Manager (CVM)
REDHAT Logical Volume Manager (LVM)
SANBOLIC LaScala
VERITAS* [Cluster] Volume Manager (CVM/VxVM)
* Merged with Symantec, July 2005
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
16
How is data shared ?
Volume level
EDUCATION
• Volume Managers allow data to be shared at a low level
(block) without usually a built-in locking mechanism
– Higher level applications control concurrent accesses to the data as
needed
– Can combine or divide physical resources (e.g., concatenation,
mirroring, striping) and share the result
– Volume Managers and the VTOC* problem
• Every OS has its own VTOC format
• Every VM has its own Volume Header and Table Definition
– Same VM everywhere and you can share raw volume or same FS
– Byte orders between processor
• Big Endian: Sparc, PA-Risc, Power – Little Endian: Intel)
– Block size on the device and block boundary
could cause troubles
– Concurrent or Serial access
* Volume Table Of Contents
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Check
Check out
out
SNIA
SNIA Tutorial:
Tutorial:
•Storage
•Storage
Virtualization
Virtualization
17
How is data shared ?
Volume level
EDUCATION
• In-Band Virtualization
• Out-of-Band Virtualization
Application
Application
Servers
Servers
Volume
allocation
Intelligent
switch and/or
Appliance
Storage
Storage
Network
Network
Appliance
Shared
Disks
Volume
creation
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Shared
Disks
18
How is data shared ?
Volume level
EDUCATION
Application
HPC App.
HPC App.
HPC App.
Servers
HPC App.
…
Shared Volume Manager - Storage Software
Storage
Network
Example:
HPC* Application
- How ?
•
•
Shared Disks
Own data format on disk
Own lock mechanism
- Benefits
•
•
Increased throughput
More effective use of servers
* High Performance Computing
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
19
How is data shared ?
File/File System level
EDUCATION
• Share at File or File System (FS) level
• Multiple approaches & levels of maturity
– Block-based
• Local (physical) Disk File System* for serial data sharing
• Disk based Cluster File System
• SAN File System or SAN File Sharing System
– File-based or Distributed File System
• NFS/CIFS, WAFS and NFM**
• Global, Parallel and Distributed FS
* like UFS, HFS, XFS, JFS, VxFS, NTFS, ext2/3…
** Network File Management (also Network File Virtualization)
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
20
File/record layer
EDUCATION
Cluster
FS
Host
Host. with LVM
Host. with LVM
and software RAID
File/record
layer
Application
Application
level
Dist. FS
LAN
SAN FS
NAS
headCluster FS
Shared
LVM
SN
Block
layer
Host
1. Direct-attach
2. SN-attach
3. NAS head
4. NAS server
NAS
server
Disk array
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Host block-aggregation
Network block-aggregation
Device block-aggregation
21
How is data shared ?
File/File System level – Local Disk File System
•
•
•
EDUCATION
Serial File System sharing on same or dissimilar OS via common
Volume Manager & Physical File System
Good for sequential (not concurrent) data processing and data
migration
If OS is different
– Same Volume Manager avoids VTOC incompatibilities
– Byte order differences may require meta-data conversion
• Intel is Little-endian, most others are Big Endian
•
Examples
– Homogeneous OS (common case)
• Most file systems (and volume managers) support this
• UFS, HFS, XFS, JFS, VxFS, ext2/3… SDS/SVM, LVM, XVM,
VxVM… Sanbolic Kayo, DNF Dynamic Share
– Heterogeneous OS (need common volume manager)
• VERITAS* Storage Foundation with Portable Data Container feature
* Merged with Symantec, July 2005
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
22
How is data shared ?
File/File System level – Local Disk File System
EDUCATION
Example:
DW* Application
OS #0
OS #1
OS #2
OS #3
- How ?
•
•
Storage
Network
•
•
•Import Disk Group
- Benefits
•Start Volume
•Mount File System
•Deport Disk Group
•Stop Volume
•Umount File System
OS #0 server stores data
OS #1 server starts
batches
OS #2 server loads data
into the DW
OS #3 server backups
data
•
•
•
Shared
Disks
•
No data multiplication
Cost effective for Storage
More effective use of
servers
No time wasted in
copying data between
servers
* Data Warehouse
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
23
How is data shared ?
File/File System level – Cluster File System
EDUCATION
• Cluster File System (also called Shared Data Cluster)
–
–
–
–
A Cluster FS allow a FS and files to be shared
All nodes understand Physical (on-disk) FS structure
The File System is mounted by all the nodes
Single FS Image
• Same data view from all nodes
– Examples
•
•
•
•
•
HP CFS (TruCluster)
HP/Cal. Soft. Monster FS
IBM GPFS
MACROIMPACT SANique CFS
POLYSERVE Matrix Server
•
•
•
REDHAT GFS 1
SANBOLIC MelioFS
VERITAS2 CFS
1 – Sistina acquired by RedHat
2 – Merged with Symantec, July 2005
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
24
How is data shared ?
File/File System level – Cluster FS
Web
Server
First
EDUCATION
HeartBeat
Web
Server
Lock Management
Host
Web
Server
Second
Host
Cluster File System
Cluster Volume Manager
Cluster
Storage
Network
Example:
Web Servers Farm
•
How ?
•
•
•
Shared VM/FS
Load Balancer in front
Shared Disks
Benefits
•
•
•
•
Increased throughput
More effective use of servers
Failure is transparent
SSI/SFSI, High SLAs
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Optional Layer
25
How is data shared ?
File/File System level – Cluster File System
EDUCATION
• Asymmetric & Symmetric Implementation
– Asymmetric uses master node for logging and locking
• Lock Mechanism
– Distributed or Global Lock Management (DLM/GLM)
• Different implementation strategies
– Granularity varies: file, record, byte…
• Cache Coherence – Single File System Image
– Every modification is seen by all nodes as soon as a
modification in the data sharing domain occurs
• Usage Consideration: Concurrent vs Serial data access
– Concurrent: multiple systems access the data simultaneously
– Serial: one system at a time uses and access the data
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
26
How is data shared ?
Advanced methods – File/FS Approaches
EDUCATION
• SAN File System
– 1 node (Master) or a set of masters
• Understand, manage and use metadata on disk
• Use of file system even if portions of it are inaccessible
• block addresses distributed to nodes (clients) on request
– Other nodes (clients)
• connection to SAN storage
• Avoid overhead due to Metadata management
• access to data directly using blocks addresses sent by Master(s)
– Designed to support hundreds or thousands of nodes
– Mixed role between direct data access with
host based thin software and NAS access
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
27
How is data shared ?
File/File System level – SAN File System
EDUCATION
– Flexibility of network FS at SAN speed
– Long-term goal for the industry development for Capacity and
Performance scaling
• « Scaling hundreds of PetaBytes of capacity and tens of GigaBytes/sec »
– More recent File Server Generation
– Examples
•
•
•
•
APPLE Xsan
ADIC StorNext FS
DataPlow SAN FS
& Nasan FS
EMC Celerra HighRoad,
MPFS/MPFSi
•
•
•
•
•
HP DirectNFS – xNFS (Cal. Soft.)
– Transoft Fibrenet
IBM TotalStorage SAN FS,
SANergy
IBRIX FusionFS
SGI CXFS
SUN QFS
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
28
How is data shared ?
File/File System level – SAN FS
EDUCATION
App.
File
Server
Data Network - LAN
Metadata
Server
App.
App.
Client sw
Client sw
File Request
NFS/CIFS
Server
Block list
Data and
Control Access
Storage
Network
Data Access
Example:
Shared
Disks
Multimedia Application
•
How ?
•
•
•
•
Benefits
•
•
•
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
1 big server with NFS/CIFS layer
Server and Client SAN FS layer
Hundreds of clients
Increased throughput
Consolidate storage, very scalable
More effective use of resources
29
How is data shared ?
File/File System level – SAN FS
EDUCATION
• How it works ?
– Asymmetric or Client/Server model
– Server controls client access, resolves conflicts
– Thin client software layer handles SAN device and server
interaction
• Lock Mechanism
–
–
–
–
Provided by the server at a central location
Various granularity: file, record, byte…
Some implementations use SMB or NFS semantics
The server needs to be protected cause it represents a SPOF
• Cache Coherency
– Some implementations deliver cache coherency with traditional
validate/invalidate mechanism, others don’t offer cache at all
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
30
How is data shared ?
File/File System level – OSD
EDUCATION
• OSD = Object-based Storage Device
• An object comprises
– Application data (e.g., file, record)
– Device-managed metadata (e.g., block allocation)
– User-accessible attributes (e.g., access times)
• Objects have file-like methods for access
– Open, close, read, write, get/set attributes
– Commands are authorized
• Industry offerings
– Lustre (www.lustre.org)
– Bull, CFS, Cray, HP, Scali, SUN… – Lustre based
– Panasas
ID x123
Blocks:3,42
Length:512
• SNIA OSD Working Group
– OSD as a SCSI command set
– www.snia.org/tech_activities/workgroups/osd
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Check
Check out
out
SNIA
SNIA Tutorial:
Tutorial:
•Object-based
•Object-based
Storage
Storage
Device
Device
31
How is data shared ?
File/File System level – OSD
EDUCATION
CPU
CPU
Applications
Applications
System Call Interface
System Call Interface
File System
User Component
File System
User Component
File System
Storage Component
Object Interface
Block Interface
File System
Storage Component
Block I/O Manager
Storage Device
Block I/O Manager
Storage Device
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
32
How is data shared ?
File/File System level – OSD
EDUCATION
Clients
A cc
es s
R eq
uest
SECRET
KEY
A
DAT
EthSAN
switch
NT
E
EM
G
NA
A
M
Managers
Intelligent Device
Space Management
Backup/Recovery
QoS via attributes
Security
SECRET
KEY
SECRET
KEY
Validate
Validate Capability
Capability
Object-based Storage Devices
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
33
How is data shared ?
File/File System level – Parallel NFS (pNFS)
⇒
Now
Client
EDUCATION
pNFS Goal
Client
Host Net
Host Net
NFS
v4
NFS
v4
Data
Storage Net
Storage Net
NFS Server
NFS Server
Data
Storage Servers
•
Storage Servers
Allow NFSv4, data to bypass NFS server
– No application changes, similar management model
•
pNFS extensions to NFSv4 communicate data location to clients
– Clients access data via Fibre Channel, iSCSI, OSD, or even NFS
•
IETF standardization in progress
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
34
How is data shared ?
File/File System level – SANFS vs CFS
Characteristics
& Features
EDUCATION
SAN FS
Cluster FS
Tolerance of Distance
(between server and
clients)
Important
Limited
# of nodes
Hundreds
Dozens
Yes
No
Yes, usually
No – cluster assigns
functions to nodes
Metadata server only
(clients may understand
if same OS)
All nodes (Cluster FS
currently requires
same OS)
Heterogeneous OS
Dedicated Meta-Data
Server(s) Required
Physical filesystem
layout knowledge
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
35
How is data shared ?
File/File System level – Parallel File System
EDUCATION
• Concept/Idea: Data is striped between servers (I/O
nodes)
• Features
– Cluster-wide consistent name space
– User control for file striping across I/O nodes
• Asymmetric (master + slave servers + clients)
– GoogleFS, PVFS*, IBRIX, Panasas (osd)
• Symmetric (peer servers + clients)
– TerraScale, Isilon, Exanet, NetApp (Spinnaker Networks)
* Parallel Virtual File System
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
36
How is data shared ?
File/File System level – Parallel File System
• Asymmetric
EDUCATION
• Symmetric
#0
#1
#2
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
37
EDUCATION
Conclusion
Conclusion
Various ways to Share Data…
EDUCATION
• Many products and philosophy in the industry
–
–
–
–
OS, disk (local) file system
Methods to protect data (locking)
Cache coherency mechanisms and semantics
Caused by varied objectives and applications
• There is no single, simple, efficient data format
available on all operating systems !! (sorry)
– Server and client software needed for Data Sharing
– Remember VTOC and Byte ordering potential issue
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
39
Conclusion
… to leverage the infrastructure
EDUCATION
• There are a number of things to consider when
choosing a file system or server
–
–
–
–
–
Will the application work as desired?
Will it perform and scale?
Does it have the required data management services?
Is it secure enough?
Is it easy to use and manage?
• There is no single solution that is superior in all
cases BUT these approaches deliver real
applications and business benefits
– Real measured ROI
– Performance, Availability and Manageability
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
40
Q&A / Feedback
EDUCATION
• Please send any questions or comments on this
presentation to SNIA: [email protected]
Many thanks to the following individuals
for their contributions to this tutorial.
SNIA Education Committee
•
•
•
•
Symantec (Philippe Nicolas)
ONStor
(Jonathan Goldick)
EMC²
(David Black)
CA, Cisco, CNT, Crossroads, EvaluatorGroup, HDS,
HGAI, Inrange, Knowledge Transfer, Microsoft,
NationWide, QLogic, Sandia National Laboratories,
Seagate, Solution Technology, Sun Microsystems &
VERITAS Software
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
41
EDUCATION
Advanced Data Sharing
Technologies
version 9
Philippe Nicolas, Symantec
Jonathan Goldick, ONStor
Advanced Data Sharing
Technologies – Part 1
version 9
Philippe Nicolas, Symantec
Jonathan Goldick, ONStor
Abstract
EDUCATION
Advanced Data Sharing Technologies – Part 1
The What, Why and How of Data Sharing technologies plus block and filebased approaches
for IT Director and Managers, IT/Storage/System Eng., Admins, Architects
and Trainers
How to deliver more performance and data accessibility with little to no
additional cost? How to take advantage of existing storage infrastructure to
provide more value to end-users and the global enterprise? A clear industry
direction indicates that Data Sharing architectures and technology can be a
good way to achieve these objectives.
The first session offers a definition of Data Sharing and a discussion of its
benefits with examples linked to the SNIA Shared Storage Model. We cover the
main data sharing approaches and describe how they can improve
performance, data accessibility, and manageability. This includes Scale-in and
Scale-out methods based on block, file and application technologies such as:
Cluster Volume Managers, SAN File Systems, Cluster File Systems, Parallel
NFS (pNFS), Object-based Storage Devices (OSD) and Global/Parallel File
System.
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
2
SNIA Legal Notice
EDUCATION
• The material contained in this tutorial is copyrighted by
the SNIA
• Member companies and individuals may use this material
in presentations and literature under the following
conditions:
– Any slide or slides used must be reproduced without modification
– The SNIA must be acknowledged as source of any material used
in the body of any document containing material from these
presentations
• This presentation is a project of the SNIA Education
Committee
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
3
Agenda
EDUCATION
• What is Data Sharing ?
– Definition
• Why Data Sharing ?
– End User Benefits
• How is Data shared ?
– Block and file-based approaches
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
4
EDUCATION
What is Data Sharing ?
Why Data Sharing ?
What is Data Sharing ?
Definition
•
EDUCATION
Shared access to
same data (value & location) by multiple systems
– Read/write: changes to data become visible to all servers
– Read-only access via mechanisms that support shared read/write
access
•
Examples
– Read/write access to a shared file is data sharing
• So is read-only access to a shared file
– Clone/Snapshots (read-only or read/write) are not data sharing
• Changes do not affect original data
•
Caching is data sharing when changes propagate
– Changes to cached data must become visible to all
– Other data changes must become visible via cache
•
Replication/CDP* is not data sharing because location changes
– Potential divergence of data
* CDP: Continuous Data Protection
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
6
Why Data Sharing ?
End User Benefits
EDUCATION
• Better performance and scalability
– Larger server can be expen$$ive
• Sharing: apply more servers to same problem
– Scales well for some applications, poorly for others
– Can avoid replication or cloning
• Concurrency and Content access distribution
– Use same data for more than one application
• Administration
– Consolidated shared resource has lower TCO
– Data Sharing increases the benefits of Storage/Data
Consolidation
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
7
Storage Consolidation
Scale-up by Scale-In
EDUCATION
File
Server
Data Network - LAN
NFS/CIFS
Server
Scale-In
Storage
Network
Shared
Disks
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
8
Performance Improvement
Scale-up by Scale-Out
EDUCATION
Application
Server
Application
…
DB Engine
Cluster Software
Shared Storage Software
Scale-out
Storage
Network
Shared Disks
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
9
Why Data Sharing ?
How to apply Data Sharing to do useful things
EDUCATION
• High Availability Clusters (local & geographic)
• Scaling applications
– Web Servers - Read mostly/load balanced
– Databases/OLTP/DW - Mostly use direct I/O
• Parallel applications and fast failover
• Systems and Applications Consolidation/Migration
• Off-host processing
– Based on shared file system
– Can also use by Point-in-Time copy techniques (not related to
our data sharing definition)
• …address both Performance and Availability with no
administration degradation and overhead
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
10
Some technologies
and products …
IBM AFS
WebNFS
ISO9660
EDUCATION
Apple Xsan
Cisco FileEngine
Coda
VERITAS CFS
RFS
DiskSites FilePort
Distributed,
Samba FineGround
PolyServe Matrix Server
Oracle OPS/RAC
ClusterLUSTRE
or
WebFS
EMC
HighRoad
IBM Storage Tank
IBRIX
FusionFS
SAN
File
System
SGI CXFS
CIFS
DB2
WAFS
ONStor STOR-FS
HP TruCluster/CFS
ADIC StorNext FS
Parallel &Sybase MPP
SUN SAM-FS
Partitioned
Sanbolic Melio FS
IBM SANergy
Applications
SMB
Tacit Networks Ishared
OSD
PVFS
Redhat GFS
PPFS
Sun QFS
Volume
WebNFS
Isilon IQ OneFS
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
OpenAFS
NFS
Informix XPS
DFS
Nuview StorageX
pNFS
11
EDUCATION
How is Data Shared ?
How is data shared ?
Approaches and methods
EDUCATION
• Several levels are possible
– Share at the volume level (block based)
– Share at the file or file system level (file, block or object* based)
– Share at the database or application level (custom)
• In all cases, all these methods could occur
– among like or dissimilar systems (OS),
– concurrently or serially,
– directly at the storage or in the network
* For OSD
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
13
How is data shared ?
Approaches and methods
EDUCATION
• Traditional/Historical
– Block level: Volume Management
– File/File System (FS) level: Local FS (serial data sharing) and
distributed methods with NAS, Samba, AFP (AppleShare), DFS,
AFS/OpenAFS, RFS, Coda…
– App./DB level: custom built-in methods (RDBMS, Email
systems…)
Check
Check out
out
SNIA
SNIA Tutorial:
Tutorial:
• Advanced/Recent - File/FS level
– Distributed: WAFS approach (NAS extension)
and Network File Management/Virtualization
(NFM), Global FS, SANFS and Cluster FS
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
•• Storage
Storage
Virtualization
Virtualization
Check
Check out
out
SNIA
SNIA Tutorial:
Tutorial:
•• NAS
NAS &
& iSCSI
iSCSI
14
How is data shared ?
The SNIA Shared Storage Model
EDUCATION
Application
Application
level
Database
(dbms)
Cluster FS
WAFS
NAS
File system
(FS)
Shared
LVM
SAN FS
Host
Network
Block
aggregation
Device
Storage
Storage devices
devices (disks,
(disks, …)
…)
Block layer
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Discovery, monitoring
monitoring
Discovery,
Resource mgmt,
mgmt, configuration
configuration
Resource
Security, billing
billing
Security,
Redundancy mgmt
mgmt (backup,
(backup, …)
…)
Redundancy
High availability
availability (fail-over,
(fail-over, …)
…)
High
Capacity planning
planning
Capacity
File/record layer
Services
Storage domain
GFS
15
How is data shared ?
Volume level
EDUCATION
LVM
LVM
Data
Path
Data Layout
and
Organization
•
Examples
–
–
–
–
–
–
–
EMC PowerPath Volume Manager (PPVM)
HP Shared Logical Volume Manager (SLVM)
IBM Logical Volume Manager (LVM)
MACROIMPACT SANique Cluster Volume Manager (CVM)
REDHAT Logical Volume Manager (LVM)
SANBOLIC LaScala
VERITAS* [Cluster] Volume Manager (CVM/VxVM)
* Merged with Symantec, July 2005
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
16
How is data shared ?
Volume level
EDUCATION
• Volume Managers allow data to be shared at a low level
(block) without usually a built-in locking mechanism
– Higher level applications control concurrent accesses to the data as
needed
– Can combine or divide physical resources (e.g., concatenation,
mirroring, striping) and share the result
– Volume Managers and the VTOC* problem
• Every OS has its own VTOC format
• Every VM has its own Volume Header and Table Definition
– Same VM everywhere and you can share raw volume or same FS
– Byte orders between processor
• Big Endian: Sparc, PA-Risc, Power – Little Endian: Intel)
– Block size on the device and block boundary
could cause troubles
– Concurrent or Serial access
* Volume Table Of Contents
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Check
Check out
out
SNIA
SNIA Tutorial:
Tutorial:
•Storage
•Storage
Virtualization
Virtualization
17
How is data shared ?
Volume level
EDUCATION
• In-Band Virtualization
• Out-of-Band Virtualization
Application
Application
Servers
Servers
Volume
allocation
Intelligent
switch and/or
Appliance
Storage
Storage
Network
Network
Appliance
Shared
Disks
Volume
creation
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Shared
Disks
18
How is data shared ?
Volume level
EDUCATION
Application
HPC App.
HPC App.
HPC App.
Servers
HPC App.
…
Shared Volume Manager - Storage Software
Storage
Network
Example:
HPC* Application
- How ?
•
•
Shared Disks
Own data format on disk
Own lock mechanism
- Benefits
•
•
Increased throughput
More effective use of servers
* High Performance Computing
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
19
How is data shared ?
File/File System level
EDUCATION
• Share at File or File System (FS) level
• Multiple approaches & levels of maturity
– Block-based
• Local (physical) Disk File System* for serial data sharing
• Disk based Cluster File System
• SAN File System or SAN File Sharing System
– File-based or Distributed File System
• NFS/CIFS, WAFS and NFM**
• Global, Parallel and Distributed FS
* like UFS, HFS, XFS, JFS, VxFS, NTFS, ext2/3…
** Network File Management (also Network File Virtualization)
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
20
File/record layer
EDUCATION
Cluster
FS
Host
Host. with LVM
Host. with LVM
and software RAID
File/record
layer
Application
Application
level
Dist. FS
LAN
SAN FS
NAS
headCluster FS
Shared
LVM
SN
Block
layer
Host
1. Direct-attach
2. SN-attach
3. NAS head
4. NAS server
NAS
server
Disk array
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Host block-aggregation
Network block-aggregation
Device block-aggregation
21
How is data shared ?
File/File System level – Local Disk File System
•
•
•
EDUCATION
Serial File System sharing on same or dissimilar OS via common
Volume Manager & Physical File System
Good for sequential (not concurrent) data processing and data
migration
If OS is different
– Same Volume Manager avoids VTOC incompatibilities
– Byte order differences may require meta-data conversion
• Intel is Little-endian, most others are Big Endian
•
Examples
– Homogeneous OS (common case)
• Most file systems (and volume managers) support this
• UFS, HFS, XFS, JFS, VxFS, ext2/3… SDS/SVM, LVM, XVM,
VxVM… Sanbolic Kayo, DNF Dynamic Share
– Heterogeneous OS (need common volume manager)
• VERITAS* Storage Foundation with Portable Data Container feature
* Merged with Symantec, July 2005
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
22
How is data shared ?
File/File System level – Local Disk File System
EDUCATION
Example:
DW* Application
OS #0
OS #1
OS #2
OS #3
- How ?
•
•
Storage
Network
•
•
•Import Disk Group
- Benefits
•Start Volume
•Mount File System
•Deport Disk Group
•Stop Volume
•Umount File System
OS #0 server stores data
OS #1 server starts
batches
OS #2 server loads data
into the DW
OS #3 server backups
data
•
•
•
Shared
Disks
•
No data multiplication
Cost effective for Storage
More effective use of
servers
No time wasted in
copying data between
servers
* Data Warehouse
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
23
How is data shared ?
File/File System level – Cluster File System
EDUCATION
• Cluster File System (also called Shared Data Cluster)
–
–
–
–
A Cluster FS allow a FS and files to be shared
All nodes understand Physical (on-disk) FS structure
The File System is mounted by all the nodes
Single FS Image
• Same data view from all nodes
– Examples
•
•
•
•
•
HP CFS (TruCluster)
HP/Cal. Soft. Monster FS
IBM GPFS
MACROIMPACT SANique CFS
POLYSERVE Matrix Server
•
•
•
REDHAT GFS 1
SANBOLIC MelioFS
VERITAS2 CFS
1 – Sistina acquired by RedHat
2 – Merged with Symantec, July 2005
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
24
How is data shared ?
File/File System level – Cluster FS
Web
Server
First
EDUCATION
HeartBeat
Web
Server
Lock Management
Host
Web
Server
Second
Host
Cluster File System
Cluster Volume Manager
Cluster
Storage
Network
Example:
Web Servers Farm
•
How ?
•
•
•
Shared VM/FS
Load Balancer in front
Shared Disks
Benefits
•
•
•
•
Increased throughput
More effective use of servers
Failure is transparent
SSI/SFSI, High SLAs
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Optional Layer
25
How is data shared ?
File/File System level – Cluster File System
EDUCATION
• Asymmetric & Symmetric Implementation
– Asymmetric uses master node for logging and locking
• Lock Mechanism
– Distributed or Global Lock Management (DLM/GLM)
• Different implementation strategies
– Granularity varies: file, record, byte…
• Cache Coherence – Single File System Image
– Every modification is seen by all nodes as soon as a
modification in the data sharing domain occurs
• Usage Consideration: Concurrent vs Serial data access
– Concurrent: multiple systems access the data simultaneously
– Serial: one system at a time uses and access the data
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
26
How is data shared ?
Advanced methods – File/FS Approaches
EDUCATION
• SAN File System
– 1 node (Master) or a set of masters
• Understand, manage and use metadata on disk
• Use of file system even if portions of it are inaccessible
• block addresses distributed to nodes (clients) on request
– Other nodes (clients)
• connection to SAN storage
• Avoid overhead due to Metadata management
• access to data directly using blocks addresses sent by Master(s)
– Designed to support hundreds or thousands of nodes
– Mixed role between direct data access with
host based thin software and NAS access
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
27
How is data shared ?
File/File System level – SAN File System
EDUCATION
– Flexibility of network FS at SAN speed
– Long-term goal for the industry development for Capacity and
Performance scaling
• « Scaling hundreds of PetaBytes of capacity and tens of GigaBytes/sec »
– More recent File Server Generation
– Examples
•
•
•
•
APPLE Xsan
ADIC StorNext FS
DataPlow SAN FS
& Nasan FS
EMC Celerra HighRoad,
MPFS/MPFSi
•
•
•
•
•
HP DirectNFS – xNFS (Cal. Soft.)
– Transoft Fibrenet
IBM TotalStorage SAN FS,
SANergy
IBRIX FusionFS
SGI CXFS
SUN QFS
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
28
How is data shared ?
File/File System level – SAN FS
EDUCATION
App.
File
Server
Data Network - LAN
Metadata
Server
App.
App.
Client sw
Client sw
File Request
NFS/CIFS
Server
Block list
Data and
Control Access
Storage
Network
Data Access
Example:
Shared
Disks
Multimedia Application
•
How ?
•
•
•
•
Benefits
•
•
•
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
1 big server with NFS/CIFS layer
Server and Client SAN FS layer
Hundreds of clients
Increased throughput
Consolidate storage, very scalable
More effective use of resources
29
How is data shared ?
File/File System level – SAN FS
EDUCATION
• How it works ?
– Asymmetric or Client/Server model
– Server controls client access, resolves conflicts
– Thin client software layer handles SAN device and server
interaction
• Lock Mechanism
–
–
–
–
Provided by the server at a central location
Various granularity: file, record, byte…
Some implementations use SMB or NFS semantics
The server needs to be protected cause it represents a SPOF
• Cache Coherency
– Some implementations deliver cache coherency with traditional
validate/invalidate mechanism, others don’t offer cache at all
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
30
How is data shared ?
File/File System level – OSD
EDUCATION
• OSD = Object-based Storage Device
• An object comprises
– Application data (e.g., file, record)
– Device-managed metadata (e.g., block allocation)
– User-accessible attributes (e.g., access times)
• Objects have file-like methods for access
– Open, close, read, write, get/set attributes
– Commands are authorized
• Industry offerings
– Lustre (www.lustre.org)
– Bull, CFS, Cray, HP, Scali, SUN… – Lustre based
– Panasas
ID x123
Blocks:3,42
Length:512
• SNIA OSD Working Group
– OSD as a SCSI command set
– www.snia.org/tech_activities/workgroups/osd
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
Check
Check out
out
SNIA
SNIA Tutorial:
Tutorial:
•Object-based
•Object-based
Storage
Storage
Device
Device
31
How is data shared ?
File/File System level – OSD
EDUCATION
CPU
CPU
Applications
Applications
System Call Interface
System Call Interface
File System
User Component
File System
User Component
File System
Storage Component
Object Interface
Block Interface
File System
Storage Component
Block I/O Manager
Storage Device
Block I/O Manager
Storage Device
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
32
How is data shared ?
File/File System level – OSD
EDUCATION
Clients
A cc
es s
R eq
uest
SECRET
KEY
A
DAT
EthSAN
switch
NT
E
EM
G
NA
A
M
Managers
Intelligent Device
Space Management
Backup/Recovery
QoS via attributes
Security
SECRET
KEY
SECRET
KEY
Validate
Validate Capability
Capability
Object-based Storage Devices
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
33
How is data shared ?
File/File System level – Parallel NFS (pNFS)
⇒
Now
Client
EDUCATION
pNFS Goal
Client
Host Net
Host Net
NFS
v4
NFS
v4
Data
Storage Net
Storage Net
NFS Server
NFS Server
Data
Storage Servers
•
Storage Servers
Allow NFSv4, data to bypass NFS server
– No application changes, similar management model
•
pNFS extensions to NFSv4 communicate data location to clients
– Clients access data via Fibre Channel, iSCSI, OSD, or even NFS
•
IETF standardization in progress
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
34
How is data shared ?
File/File System level – SANFS vs CFS
Characteristics
& Features
EDUCATION
SAN FS
Cluster FS
Tolerance of Distance
(between server and
clients)
Important
Limited
# of nodes
Hundreds
Dozens
Yes
No
Yes, usually
No – cluster assigns
functions to nodes
Metadata server only
(clients may understand
if same OS)
All nodes (Cluster FS
currently requires
same OS)
Heterogeneous OS
Dedicated Meta-Data
Server(s) Required
Physical filesystem
layout knowledge
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
35
How is data shared ?
File/File System level – Parallel File System
EDUCATION
• Concept/Idea: Data is striped between servers (I/O
nodes)
• Features
– Cluster-wide consistent name space
– User control for file striping across I/O nodes
• Asymmetric (master + slave servers + clients)
– GoogleFS, PVFS*, IBRIX, Panasas (osd)
• Symmetric (peer servers + clients)
– TerraScale, Isilon, Exanet, NetApp (Spinnaker Networks)
* Parallel Virtual File System
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
36
How is data shared ?
File/File System level – Parallel File System
• Asymmetric
EDUCATION
• Symmetric
#0
#1
#2
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
37
EDUCATION
Conclusion
Conclusion
Various ways to Share Data…
EDUCATION
• Many products and philosophy in the industry
–
–
–
–
OS, disk (local) file system
Methods to protect data (locking)
Cache coherency mechanisms and semantics
Caused by varied objectives and applications
• There is no single, simple, efficient data format
available on all operating systems !! (sorry)
– Server and client software needed for Data Sharing
– Remember VTOC and Byte ordering potential issue
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
39
Conclusion
… to leverage the infrastructure
EDUCATION
• There are a number of things to consider when
choosing a file system or server
–
–
–
–
–
Will the application work as desired?
Will it perform and scale?
Does it have the required data management services?
Is it secure enough?
Is it easy to use and manage?
• There is no single solution that is superior in all
cases BUT these approaches deliver real
applications and business benefits
– Real measured ROI
– Performance, Availability and Manageability
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
40
Q&A / Feedback
EDUCATION
• Please send any questions or comments on this
presentation to SNIA: [email protected]
Many thanks to the following individuals
for their contributions to this tutorial.
SNIA Education Committee
•
•
•
•
Symantec (Philippe Nicolas)
ONStor
(Jonathan Goldick)
EMC²
(David Black)
CA, Cisco, CNT, Crossroads, EvaluatorGroup, HDS,
HGAI, Inrange, Knowledge Transfer, Microsoft,
NationWide, QLogic, Sandia National Laboratories,
Seagate, Solution Technology, Sun Microsystems &
VERITAS Software
SNIA Advanced Data Sharing Technologies Tutorial (v9) - Storage Networking World USA - April 3-6th, 2006
© 2005-2006 Storage Networking Industry Association. All Rights Reserved.
41
EDUCATION
Advanced Data Sharing
Technologies
version 9
Philippe Nicolas, Symantec
Jonathan Goldick, ONStor
