虛擬文件系統(tǒng)

1、Linux* Virtual File SystemIntel China Software Center* Other names and brands may be claimed as the property of others2022/4/292AgendaLinux* file system overviewVFS data structuresPath lookupSystem callsA simple FS implementation* Other names and brands may be claimed as the property of others2022/4

2、/293Linux* file systemsVirtual File System like other UnixLinux* supports many file system typesEach type is a different filesystem implementationUsually in a loadable moduleLinux* supports many file system instancesEach instance is a mounted file systemEvery system has a root file system/ /hd0hd1*

3、Other names and brands may be claimed as the property of others2022/4/294Linux* kernel file system architectureVFS layerext3ntfsvfatnfssmbprocsysfsBlock device layernetwork layerGeneric file system interface* Other names and brands may be claimed as the property of others2022/4/295Linux* file system

4、 typesBlock device-based file systemsBuild on top of a block deviceExt3, ntfsNetwork-based file systemsBuild on top of network protocolsNfs, smbVirtual/pseudo file systemsBuild in memoryProc, sysfs* Other names and brands may be claimed as the property of others2022/4/296VFS layerObject-orientedUnif

5、ied file system modelTraditional Unix likeFS implementation should be converted to the model if necessaryPurposeUnified user interfaceAbstraction for all different file system implementationsResponsibilitiesSystem callsManage file systems data structuresInteract with different filesystem implementat

6、ions2022/4/297VFS objectsImportant: hard disk VS. kernelSuperblockInfo about a file system instanceMay have disk counterpartInodeInfo about a fileMay have disk counterpartDentryabout directory tree or about a name pathFileabout an opened file2022/4/298Disk VS kernelsuper blockinode blocksdata blocks

7、dentryinodefilesuper blockdentrydentryinodekerneldisk2022/4/299VFS objectsDisk filesuperblock objectInode objectdentryobjectdentryobjectfile objectfile objectprocess 1process 2process 3file objectfdf_dentryd_inodei_sb2022/4/2910VFS objects common groundReference counter for each objectShare (open an

8、 object several times)State transitionCacheCache unused objects in memoryFor performance considerationAbstract objectsThe object might not equal to the corresponding one of specific fs implementation (disk)Specific fs implementation should convert their disk one to kernel oneObject and its operation

9、sFor each objectClassical object-oriented2022/4/2911VFS object methodsAn operation table for each objectFunction pointersFile system specific or maybe object specificLink VFS with specific file system implementationImplemented by specific file systemCalled in VFS2022/4/2912SummaryConcepts: VFS, fs r

10、oot, system root Object oriented methodVFS objects: disk vs. kernel2022/4/2913AgendaLinux* file system overviewVFS data structuresPath lookupSystem callsA simple FS implementation* Other names and brands may be claimed as the property of others2022/4/2914File system typeAbout a file system implement

11、ationstruct file_system_typeMultiple fs instances per file_system_type Main fieldsName (fs name, like ext3)Fs_flags (flags, like FS_REQUIRES_DEV)Get_sb (called when mounting a new fs instance)Kill_sb (called when unmounting a fs instance)Fs_supers(link to a list for all fs instances of the type)Owne

12、r(the module holding the fs implementation)2022/4/2915File system typesnameGet_sbKill_sbnextFs_supersnameGet_sbKill_sbnextFs_superss_instancess_instancess_instancesfile_systemsstruct fs_system_typestruct super“ext3”ext3_get_sb2022/4/2916Register a file systemFile system implementation must register/

13、unregister it into kernelUsually in a module init/exit!= mount a file systemregister_filesystemunregister_filesystemget_sb (superblock reading function) is the key2022/4/2917Superblock objectInformation for a fs instanceStruct superblockMaybe in disk. Kernel has a cached one for an instanceMain fiel

14、dsS_bdev block deviceS_type file system typeS_root dentry of root directoryS_op superblock operationsS_inodes, s_dirty, s_io inode listsS_blocksize, s_maxbytes parametersS_magic magic number of the fsS_count reference count2022/4/2918Superblock operationsstruct super_operationswrite_super(read super

15、 is in filesystem type)read_inode/write_inode (read/write an inode from/to disk)dirty_inode (makes an inode dirty, so it can be written back to disk)delete_inode (delete the inode from disk)2022/4/2919Inode objectInode represents a file, store files infoInode resides in disk, and kernel has a cached

16、 oneInode number is unique in one fs instanceInode cache. Recently accessed unused inode will be cached for performance purpose2022/4/2920Inode structureStruct inodeMain fieldsi_ino, i_sb(inode number & superblock)i_mapping(address space)i_fop(File operations)i_op(Inode operations)i_list, i_hash

17、, i_sb_list (lists)i_uid, i_gid, i_mod,i_atime(attributes)i_count (reference count)2022/4/2921Inode listsi_listinode_in_use (in use but not dirty, i_count 0)inode_unused (unsed, i_count = 0. unused inode isnt freed immediately)super_block-s_dirty (dirty, i_count 0)i_sb_listsuper_block-s_inodesi_hash

18、Fast search a cached inodeinode_hashtableHashed by i_ino & i_sb2022/4/2922Inode listsi_hashi_listi_hashi_listi_hashi_listinode_in_useinode_unuseds_dirtystruct super_blockInode_hashtablestruct inodes_inodesi_hashi_listi_sb_listi_sb_listi_sb_listi_sb_list2022/4/2923Inode operationsstruct inode_ope

19、rationslookup (lookup an inode in a parent dir)create (create a new file)mkdir/rmdir(create/delete dir)Link/unlink/symlink (create hard link/symbol link)Inode kernel APIsIget(), ilookup(), 2022/4/2924Dentry objectKernel data structure for directory entryKernel cache, no disk counterpart File name to

20、 inode mappingMultiple dentries may point to one inodeDentry cache. Recently accessed ununsed dentry will be cached for performance purposeExample: /xxx/yyy (3 dentries, /, xxx, yyy)2022/4/2925Dentry structureStruct dentryMain fieldsD_inode(its inode)D_parent, d_child, d_subdirs, d_alias, d_lru (lin

21、ks)D_nameD_mounted(if there are fs mounted)D_op (dentry operations)2022/4/2926Dentry statesfreeInused_alias links to inode-i_dentryUnusedd_lru links to dentry_unusedNegativeNo inode2022/4/2927Dentry listTree layout(d_parent, d_subdirs, d_child)Inuse list(d_alias)Unused list(d_lru)Hashtable(hash coll

22、ision list, d_hash)2022/4/2928Dentry tree layoutd_parentd_childd_subdirsd_parentd_childd_subdirsd_parentd_childd_subdirs2022/4/2929Dentry listsd_inoded_hashd_lrud_aliasd_inoded_hashd_lrud_aliasd_inoded_hashd_lrud_aliasd_inoded_hashd_lrud_aliasdentry_unusedi_dentrystruct inodestruct dentrydentry_hash

23、table2022/4/2930Dentry operationsDentry_operationsd_hash (return hash value of a file name)d_compare (compare if file name and dentry match)d_delete(delete a dentry, when d_count = 0)Dentry kernel APIsd_add(), d_alloc(), d_lookup() 2022/4/2931File objectDescribes how a process interacts with an open

24、ed fileResides in kernelStruct filef_op (file operations)f_dentry (files dentry)f_uid/f_gid/f_mode/f_owner (file attributes)fu_list (link to superblock, remount should check opened files)f_pos (read/write position)f_count (reference count)2022/4/2932File operationsStruct file_operationsOpenwrite/rea

25、dllseek (set file read/write position)mmap (called when doing mmap syscall)release (called when closing a file)poll (called when doing select/poll syscall)ioctl2022/4/2933VFS objects relationships_roots_inodess_filesi_sbi_sb_listi_dentryd_aliasd_sbd_inodefu_listf_dentrystruct super_blockstruct inode

26、struct filestruct dentry2022/4/2934Structures associated with a processStruct task_struct has two fields related with fsstruct fs_struct *fsstruct files_struct *filesstruct files_structAbout opened files of a taskImportant fields: Struct file array (fd file mapping)To get an open file: files_struct-

27、fdi, i is the file handlerStruct fs_structAbout tasks filesystem environmentRoot mountpoint (vfsmount), root dentryPwd mountpoint (vfsmount), pwd dentry2022/4/2935SummaryObjects and their data structures/operationsReferenceKernel src/include/fs.hKernel src/fs/filesystems.c (struct file_system_type)K

28、ernel src/fs/super.c (struct super)Kernel src/fs/inode.c (struct inode)Kernel src/fs/dcache.c (struct dentry)2022/4/2936AgendaLinux* file system overviewVFS data structuresPath lookupSystem callsA simple FS implementation* Other names and brands may be claimed as the property of others2022/4/2937Pat

29、hname lookupFind the inode/dentry from a given pathnameFrequently usedStart point dentryPathname starts /: fs-rootElse: fs-pwdSpecial handlingSymbolic links (check loops)Mount pointsAccess permission2022/4/2938Path_walkGet paths start point (root, pwd) and its mountpointParse one name one loopCheck

30、access permission(vfs_permission)Calculate name hashCheck special file (eg. “.”)Check dentry cache (_do_lookup)Or read inode from disk (real_lookup)Check mount point, symbolic link2022/4/2939Lookup in dcache (dentry cache)_do_lookupGet hash collision list (d_hash, parent and file name as hash key (p

31、arent-d_op-d_hash()Scan the list to get one dentryParent-d_op-d_compare(), match file name2022/4/2940Lookup in diskWhen dentry cache lookup failsreal_lookupAllocate a dentry(d_alloc()Parent-i_op-lookup(), which reads inode from disk. This is a fs specific implementation.2022/4/2941Lookup examplestru

32、ct dentry *ext3_lookup(struct inode * dir, struct dentry *dentry, struct nameidata *nd) /* get inode number */inode = iget(dir-i_sb, ino);return d_splice_alias(inode, dentry);2022/4/2942Lookup exampleiget (Get inode of inode number)Search inode cacheOr allocate a new inode (sb-s_op-alloc_inode(), fi

33、le system specific)Read inode in (sb-s_op-read_inode(), file system specific)d_splice_alias (link dentry with inode)Add dentry to inodes listAdd dentry into hash list (so it can be found by cached lookup)2022/4/2943Path walk special handlingMount point follow_mountFs mount is stackedIts possible to

34、mount multiple fs in the same directory jump to the latest mounted file system on the dentry. follow_dotdotSymbolic linkSpecific fs must support symbolic linkDo_follow_link (check loop, jump to linked file. Uses inode operations follow_link(), put_link()2022/4/2944Path walk exampledentry is found at

35、 nd with a success path walkExamplestruct nameidata nd;path_walk(name, &nd); /* do something with nd-dentry */path_release(&nd);2022/4/2945SummaryHow kernel parses a file nameThe example how kernel uses objects operationsReference:Kernel src/fs/namei.c2022/4/2946AgendaLinux* file system over

36、viewVFS data structuresPath lookupSystem callsA simple FS implementation* Other names and brands may be claimed as the property of others2022/4/2947Mount a filesystemMounting a filesystemInitialize a fs instanceGraft the root of a fs to root fss leafSpecial fs instances might not be in root fsTermin

37、ologiesMount pointRoot directory of a mounted fsRoot fs2022/4/2948Struct vfsmountMain fieldsMnt_root (root dentry of the mounted fs)Mnt_sb (superblock of mounted fs)Mnt_mountpoint(dentry of mountpoint)Mnt_parent(parent vfsmount)Mnt_hash, mnt_child, mnt_mounts (variant lists)2022/4/2949Vfsmount lists

38、mnt_mountsmnt_childmnt_hashmnt_mountsmnt_childmnt_hashmnt_mountsmnt_childmnt_hashmount_hashtable(parent_vfsmount, mountpoint_dentry)strut vfs_mount2022/4/2950sys_mountsys_mountpath_lookup (get mountpoint dentry)do_kern_mount (initialize the fs instance) Allocate vfsmount and fill it file_system_type

39、-get_sb() get superblockdo_add_mount(graft the instance into root tree) Follow down mountpoint (a dentry can mount multiple fs) graft_tree (add vfsmount to proper lists)2022/4/2951Open system callsys_openGet an empty fdPath walk to get the dentry of a file name (open_namei)Open the dentry Get a file

40、 struct and initialize it per its dentry (such as initializing fops) Call fops-openfd_install (link fd with file)2022/4/2952Struct address_spacepresents a files pagesLinkProcess address_spaceAddress_space pageStruct address_spacestruct inode*host;struct radix_tree_rootpage_tree;struct prio_tree_root

41、i_mmap;struct address_space_operations *a_ops;Struct address_space_operations.read_page/.write_page.set_page_dirty2022/4/2953Read system callsys_readGet file struct per fd (fget_light)Check access permissionfile-f_op-read()Update file read/write position counterread/write data cache (buffer) for per

42、formancef_op-read does the real file readCould direct read disk hereBut usually relies on address_spaces .read_page method.Kernel provides helper functions (generic_file_read) which have buffer handlinggeneric_file_read (abbreviated steps)Sanity checkCheck if the data is in page cache, if yes, jump

43、to last stepRead page in - Address_space-a_ops-read_page()Add read page into page cacheCopy data to userspace2022/4/2954Write system callsys_writeSimilar steps as sys_readfile-f_op-write() do real file writefile-f_op-write()Could write data to disk directlyKernel helper generic_file_writegeneric_fil

44、e_writeKernel will do buffer, usually just copy the data from userspace to kernel and mark the page and its inode dirty.Relies on address_space-a_ops-prepare_write() (prepare buffer, read some data in ) and address_space-a_ops-commit_write()(dirty data) Dirty page (a_ops-dirty_page() Dirty inode (su

45、perblock operations dirty_inode()Page writeback writes the changes to disk finally. Physically write inode to disk (super operation-write_inode() Physically write page to disk (a_ops-write_pages/write_page()2022/4/2955SummaryData structures about mount pointImplementation of typical fs related syste

46、m callsHow kernel uses objects operationsReferenceKernel src/fs/namespace.cKernel src/fs/open.cKernel src/fs/read_write.c2022/4/2956AgendaLinux* file system overviewVFS data structuresPath lookupSystem callsA simple FS implementation* Other names and brands may be claimed as the property of others20

47、22/4/2957A simple FS implementation - romfsA readonly file system implementationVery simple (less than 1000 lines code)FS format document can be found at (kernelsource/Documentation/filesystems/romfs.txt)Code can be found at (kernelsource/fs/romfs)2022/4/2958Romfs file_system_typestatic struct file_

48、system_type romfs_fs_type = .owner= THIS_MODULE,.name= romfs,.get_sb= romfs_get_sb,.kill_sb= kill_block_super,.fs_flags= FS_REQUIRES_DEV,;static int _init init_romfs_fs(void)err = register_filesystem(&romfs_fs_type);module_init(init_romfs_fs)2022/4/2959Romfs file_system_typeromfs_get_sbUses help

49、er routine (get_sb_bdev) to open block device, allocate super block struct and initialize itGet sector 0 (sb_bread) from block device. Sector 0 has super block infodo sanity check (magic number )get root inodes inode number (in super block)Get inode struct (iget) s_op-read_inode() to read inode inInitialize root dentry “/” (d_alloc_root)kill_block_superC