Check-in [3812109bfc]

Not logged in
Home TOC Leaves Timeline Branches Tags Tickets Wiki Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
SHA1 Hash:3812109bfcf52d99745fc33bd12d34ccea99fd02
Date: 2010-03-10 19:37:13
User: stephan
Comment:minor tinkering and doc updates.
Tags And Properties
Changes
hide diffs unified diffs patch

Changes to include/wh/whio/whio_epfs.h

Old (bf853fe24bc90198) New (3aba8bcf9cd4e359)
1 #ifndef WANDERINGHORSE_NET_WHIO_EPFS_H_INCLUDED 1 #ifndef WANDERINGHORSE_NET_WHIO_EPFS_H_INCLUDED
2 #define WANDERINGHORSE_NET_WHIO_EPFS_H_INCLUDED 2 #define WANDERINGHORSE_NET_WHIO_EPFS_H_INCLUDED
3 #include "whio_dev.h" /* core whio_dev API. */ 3 #include "whio_dev.h" /* core whio_dev API. */
4 #include "whio_encode.h" /* for whio_sizeof_encoded_xxx */ 4 #include "whio_encode.h" /* for whio_sizeof_encoded_xxx */
5 #include "whio_epfs_config.h" /* EPFS compile-time config options. */ 5 #include "whio_epfs_config.h" /* EPFS compile-time config options. */
81 hidden lines
87 87
88 - An EFS can add data blocks on demand, but cannot currently be 88 - An EFS can add data blocks on demand, but cannot currently be
89 shrunk. It needs a vaccuum-like feature. 89 shrunk. It needs a vaccuum-like feature.
90 90
91 91
92 | 92 <b>EPFS vs. whefs</b>
93 @section page_whio_epfs_main_whefs whio_epfs vs. whefs |
94 93
95 whio_epfs is a spin-off/refactoring of the whefs project: 94 whio_epfs is a spin-off/refactoring of the whefs project:
96 95
97 http://code.google.com/p/whefs/ 96 http://code.google.com/p/whefs/
98 97
15 hidden lines
114 113
115 - It is structured to allow stack allocation of most of its data, to 114 - It is structured to allow stack allocation of most of its data, to
116 help client code reduce calls to malloc(). 115 help client code reduce calls to malloc().
117 116
118 117
119 @section whio_epfs_main_concepts EFS Concepts | 118 <b>Concepts Overview</b>
120 119
121 Here is an overview of the core concepts encapsulated by this 120 Here is an overview of the core concepts encapsulated by this
122 embedded (pseudo-)filesystem (EFS) API: 121 embedded (pseudo-)filesystem (EFS) API:
123 122
124 - The whio_epfs class manages a single EFS container file. 123 - The whio_epfs class manages a single EFS container file.
13 hidden lines
138 waiting to be used. The inode ID 0 is reserved as the not-an-inode 137 waiting to be used. The inode ID 0 is reserved as the not-an-inode
139 sentry value. 138 sentry value.
140 139
141 - Each inode is associated with 0 or more data blocks. All blocks are 140 - Each inode is associated with 0 or more data blocks. All blocks are
142 the same size, as determined when the EFS is created (via 141 the same size, as determined when the EFS is created (via
143 whio_epfs_mkfs2()). Each inode records the ID of its initial data block | 142 whio_epfs_mkfs()). Each inode records the ID of its initial data block
144 also records its virtual size (which may span a partial block). 143 also records its virtual size (which may span a partial block).
145 144
146 - Each data block contains a few bytes of bookkeeping and up to N 145 - Each data block contains a few bytes of bookkeeping and up to N
147 bytes of user data (as defined via whio_epfs_mkfs2()). | 146 bytes of user data (as defined via whio_epfs_mkfs()).
148 147
149 - Clients open inodes as whio_dev objects, and then access them via 148 - Clients open inodes as whio_dev objects, and then access them via
150 the whio_dev API. Thus they can be used in any generic algorithms 149 the whio_dev API. Thus they can be used in any generic algorithms
151 written for that API, e.g. whio_dev_writef(). They can be used with 150 written for that API, e.g. whio_dev_writef(). They can be used with
152 the whio_stream interface by using whio_stream_for_dev(). 151 the whio_stream interface by using whio_stream_for_dev().
153 152
154 */ | 153 **/
155 154
156 /** @page page_whio_epfs_layout whio_epfs filesystem layout 155 /** @page page_whio_epfs_layout whio_epfs filesystem layout
157 156
158 A whio_epfs filesystem (EFS) is contained in a so-called container 157 A whio_epfs filesystem (EFS) is contained in a so-called container
159 file. The container file has the following internal layout: 158 file. The container file has the following internal layout:
12 hidden lines
172 FS OPTIONS: see the whio_epfs_fsopt type. 171 FS OPTIONS: see the whio_epfs_fsopt type.
173 172
174 FS HINTS: some internal hints for the EFS to help optimize certain 173 FS HINTS: some internal hints for the EFS to help optimize certain
175 operations (see whio_epfs::hints). 174 operations (see whio_epfs::hints).
176 175
177 INODES TABLE: each inode (EFS entry) contains the following data: | 176 INODES TABLE: each inode (EFS entry) contains the following data (not necessarily
| 177 in this order):
178 178
179 - The numeric ID of the inode, staring with 1. | 179 - The numeric ID of the inode, staring with 1. (ID 0="not an inode")
| 180
180 - Timestamp, in Unix-epoch format, of the last change to the inode. 181 - Timestamp, in Unix-epoch format, of the last change to the inode.
181 - The logical size of the file, in bytes. | 182
182 - ID of the first data block. | 183 - The virtual size of the pseudofile, in bytes.
| 184
| 185 - ID of the first data block, or 0 if (size==0).
| 186
183 - Internal flags. 187 - Internal flags.
184 188
185 The inodes table is whio_epfs::fsopt.inodeCount entries long. | 189 - IDs of the next and previous free inodes (or 0 if this inode is
| 190 used). The next/previous free inode IDs are used for managing a
| 191 linked list so that we can allocate new inodes to the client from
| 192 the free-list O(1) time.
| 193
| 194 The inodes table is whio_epfs::fsopt.inodeCount entries long, and
| 195 each entry takes up whio_epfs_sizeof_inode bytes in the storage.
186 196
187 The BLOCKS TABLE contains a list of metadata and client data. Each 197 The BLOCKS TABLE contains a list of metadata and client data. Each
188 block contains: 198 block contains:
189 199
190 - The numeric ID of the block, staring with 1. 200 - The numeric ID of the block, staring with 1.
191 - ID of next block in the chain 201 - ID of next block in the chain
192 - Internal flags. 202 - Internal flags.
193 - Client data 203 - Client data
> 204 - ID of next free block (or 0 if this block is used). See the notes
> 205 above regarding inode free-list links.
194 206
195 The client data part of each block is whio_epfs::fsopt::blockSize | 207 The client-data part of each block is whio_epfs::fsopt::blockSize
196 bytes long. If whio_epfs::fsopt::maxBlocks is not 0 then a EFS 208 bytes long. If whio_epfs::fsopt::maxBlocks is not 0 then a EFS
197 will not be allowed to expand beyond that many blocks, otherwise 209 will not be allowed to expand beyond that many blocks, otherwise
198 it may grow to an arbitrary number of blocks (within the numeric 210 it may grow to an arbitrary number of blocks (within the numeric
199 limits of whio_size_t). | 211 limits of whio_epfs_id_t).
200 212
201 The block and inode ID 0 is reserved for "not a block" resp. "not 213 The block and inode ID 0 is reserved for "not a block" resp. "not
202 an inode." 214 an inode."
> 215
> 216 The blocks and inodes are structured as a linked list (blocks,
> 217 singly, and inodes, doubly). These links do not imply any sort of
> 218 relationship between the object, except that we use it to arrange
> 219 all "unused" objects in a list so that we can dole then out later
> 220 on.
> 221
> 222 Initially (just after EFS creation), all blocks/inodes are
> 223 "unused", ready to be doled out to the client. As blocks/inodes
> 224 are allocated to the client, the free-list link(s) is (are)
> 225 modified to "remove" the object from the free-list. The managing
> 226 whio_epfs object must simply know the ID of the first object in
> 227 the free-list in order to allocate, in O(1) time, the
> 228 object. Managing the links adds a small i/o factor to that O(1),
> 229 but not much: at most, read/modify/write of 1 other blocks or up
> 230 to 2 other inodes.
> 231
> 232 Most management is done directly at the head of the list (the
> 233 exception being inodes opened via an explicit ID, which require
> 234 manipulation later in the chain), and the free-lists operate is
> 235 LIFO order. The most recently "freed" object will be the next one
> 236 allocated. This is part of what makes the list processing quite
> 237 cheap and the lookup O(1). When the EFS is constructed, the
> 238 initialization process goes out of its way to ensure that the
> 239 initial list is ordered from lowest to highest record ID.
203 */ 240 */
204 241
205 /** @page page_whio_epfs_memcosts whio_epfs memory costs 242 /** @page page_whio_epfs_memcosts whio_epfs memory costs
206 243
207 Here are some notes about the memory costs of whio_epfs, mainly 244 Here are some notes about the memory costs of whio_epfs, mainly
208 for use in finding good values for use with 245 for use in finding good values for use with
209 whio_epfs_mempool_setup()... 246 whio_epfs_mempool_setup()...
> 247
> 248 For typical use, where only a couple handles are opened at once
> 249 and data block chains for the opened handles do not grow really
> 250 long, a whio_epfs object can get by with less than half a
> 251 kilobyte(!) of memory in use at any given time.
210 252
211 An opened EFS which has not opened any handles takes up 253 An opened EFS which has not opened any handles takes up
212 sizeof(whio_epfs) bytes (which is, as of this writing, smaller 254 sizeof(whio_epfs) bytes (which is, as of this writing, smaller
213 than sizeof(FILE)). They do not dynamically allocate any memory 255 than sizeof(FILE)). They do not dynamically allocate any memory
214 until they have to open a handle. 256 until they have to open a handle.
215 257
216 Each opened handle costs approximately: 258 Each opened handle costs approximately:
217 259
218 - sizeof(whio_epfs_handle) 260 - sizeof(whio_epfs_handle)
219 261
220 - plus (sizeof(whio_epfs_block)*N), where N is approximately equal | 262 - plus (sizeof(whio_epfs_block)*N), where N is approximately
221 to the number of blocks owned by that handle. N may be larger | 263 equal to the number of blocks owned by that handle. N may be
222 than the current block count due to | 264 larger than the current block count due to
223 pre-allocation/reservation. The blocks list is not allocated 265 pre-allocation/reservation. The blocks list is not allocated
224 until the blocks are actually read from or written to. Simply 266 until the blocks are actually read from or written to. Simply
225 opening an inode will not allocate its block chain. | 267 opening an inode will not allocate its block chain. There is no
| 268 client-side mechanism to determine the number of blocks
| 269 associated with an inode, but it can be calculated by dividing
| 270 the inode's size by the fs' block size.
226 271
227 When multiple handles are opened for the same inode: 272 When multiple handles are opened for the same inode:
228 273
229 - One list of blocks is shared amongst them. 274 - One list of blocks is shared amongst them.
230 275
231 - Closing the handle may not free its memory immediately - it 276 - Closing the handle may not free its memory immediately - it
232 cannot be destroyed until the last opened handle pointing to it 277 cannot be destroyed until the last opened handle pointing to it
233 is closed. | 278 is closed. This is a side-effect of how handle sharing is
234 | 279 implemented (but i'd like to replace this model eventually).
235 280
236 Using whio_epfs_dev_open() opens a handle (with the above costs) 281 Using whio_epfs_dev_open() opens a handle (with the above costs)
237 and allocates about sizeof(whio_dev)+(about)20 bytes for its 282 and allocates about sizeof(whio_dev)+(about)20 bytes for its
238 implementation data. 283 implementation data.
239 <
240 For typical use, where only a few handles are opened at once and <
241 block chains do not grow insanely long, a whio_epfs object can get <
242 by with less than half a kilobyte of memory in use at any given <
243 time. If it runs out of memory, it can be configured to fall back <
244 to malloc(), realloc(), and free(), or to return allocation <
245 (out-of-memory) errors. <
246 <
247 */ 284 */
248 285
249 /** @page page_whio_epfs_conventions whio_epfs Conventions 286 /** @page page_whio_epfs_conventions whio_epfs Conventions
250 287
251 288
252 Naming conventions: | 289 <b>Naming conventions:</b>
253 290
254 - Types and free functions are named in lower_case_with_underscore. 291 - Types and free functions are named in lower_case_with_underscore.
255 292
256 - Struct member function pointers typically use lower_with_underscore, 293 - Struct member function pointers typically use lower_with_underscore,
257 but this is not a rule. 294 but this is not a rule.
11 hidden lines
269 306
270 - Public API header files live in <wh/whio/...> and should follow the 307 - Public API header files live in <wh/whio/...> and should follow the
271 same naming conventions as for Types. Private header files may be 308 same naming conventions as for Types. Private header files may be
272 named or located wherever the implementor prefers. 309 named or located wherever the implementor prefers.
273 310
274 ======================================================================== | 311
275 Pointer-to-pointer parameters for functions with flexible allocation | 312 <b>Pointer-to-pointer parameters for functions with flexible allocation
276 rules: | 313 rules:</b>
277 314
278 Many functions in the API which deal with object initialization allow 315 Many functions in the API which deal with object initialization allow
279 the caller to either provide his own memory or to allow the library to 316 the caller to either provide his own memory or to allow the library to
280 allocate it for him. Such functions have a signature which looks 317 allocate it for him. Such functions have a signature which looks
281 something like: 318 something like:
5 hidden lines
287 Most functions using this convention allow both of these calling options: 324 Most functions using this convention allow both of these calling options:
288 325
289 Client-allocated object (stack or otherwise): 326 Client-allocated object (stack or otherwise):
290 327
291 @code 328 @code
292 some_type st; | 329 some_type st = { ... sane values ... };
293 some_type * ptr = &st; 330 some_type * ptr = &st;
294 int rc = init_func( &ptr, ... ); 331 int rc = init_func( &ptr, ... );
295 @endcode 332 @endcode
296 333
297 In that case, such functions will assume that the st object was 334 In that case, such functions will assume that the st object was
11 hidden lines
309 @code 346 @code
310 some_type * ptr = NULL; 347 some_type * ptr = NULL;
311 int rc = init_func( &ptr, ... ); 348 int rc = init_func( &ptr, ... );
312 @endcode 349 @endcode
313 350
314 In this case, the library will (somehow) allocate a some_type object | 351 In this case, the library will (somehow try to) allocate a some_type
315 and assign *ptr to that (on success - it does not do so on error). | 352 object and assign *ptr to that (on success - it does not do so on
316 The caller takes over ownership and must destroy the object as | 353 error). The caller takes over ownership and must destroy the object
317 described in the documentation for init_func(). | 354 as described in the documentation for init_func().
318 |
319 355
320 The reasons such routines are set up this way, as opposed to simply 356 The reasons such routines are set up this way, as opposed to simply
321 returning a pointer to the created resource, are: 357 returning a pointer to the created resource, are:
322 358
323 - A core project goal is keeping memory costs low and reducing the 359 - A core project goal is keeping memory costs low and reducing the
147 hidden lines
471 WHIO_EPFS_MODE_FLAG_CREATE = 0x10, 507 WHIO_EPFS_MODE_FLAG_CREATE = 0x10,
472 /** 508 /**
473 Convenience equivalent of (WHIO_EPFS_MODE_WRITE | WHIO_EPFS_MODE_FLAG_CREATE). 509 Convenience equivalent of (WHIO_EPFS_MODE_WRITE | WHIO_EPFS_MODE_FLAG_CREATE).
474 */ 510 */
475 WHIO_EPFS_MODE_RWC = WHIO_EPFS_MODE_WRITE | WHIO_EPFS_MODE_FLAG_CREATE, 511 WHIO_EPFS_MODE_RWC = WHIO_EPFS_MODE_WRITE | WHIO_EPFS_MODE_FLAG_CREATE,
476 /* | 512 /**
477 Not yet used. | 513 Not yet implemented.
478 514
479 Equivalent of O_EXCL (meaning fail if inode is already used). 515 Equivalent of O_EXCL (meaning fail if inode is already used).
480 */ 516 */
481 WHIO_EPFS_MODE_FLAG_FAIL_IF_USED = 0x20, 517 WHIO_EPFS_MODE_FLAG_FAIL_IF_USED = 0x20,
482 /** 518 /**
325 hidden lines
808 844
809 fs->offsets[whio_epfs_index_fsopt] == offset into EFS of whio_epfs_fsopt. 845 fs->offsets[whio_epfs_index_fsopt] == offset into EFS of whio_epfs_fsopt.
810 846
811 fs->sizes[whio_epfs_index_fsopt] == encoded size of whio_epfs_fsopt. 847 fs->sizes[whio_epfs_index_fsopt] == encoded size of whio_epfs_fsopt.
812 */ 848 */
813 whio_epfs_index_fsopt = 1, | 849 whio_epfs_index_fsopt,
814 /** whio_epfs_hints entry. 850 /** whio_epfs_hints entry.
815 851
816 852
817 fs->offsets[whio_epfs_index_hints] == offset into EFS of internal fs hints. 853 fs->offsets[whio_epfs_index_hints] == offset into EFS of internal fs hints.
818 854
819 fs->sizes[whio_epfs_index_hints] == encoded size of internal fs hints. 855 fs->sizes[whio_epfs_index_hints] == encoded size of internal fs hints.
820 */ 856 */
821 whio_epfs_index_hints = 2, | 857 whio_epfs_index_hints,
822 /** EFS label entry. 858 /** EFS label entry.
823 <
824 Not yet used. <
825 859
826 fs->offsets[whio_epfs_index_label] == offset into EFS of label string. 860 fs->offsets[whio_epfs_index_label] == offset into EFS of label string.
827 861
828 fs->sizes[whio_epfs_index_label] == maximum encoded size of label string. 862 fs->sizes[whio_epfs_index_label] == maximum encoded size of label string.
829 */ 863 */
830 whio_epfs_index_label = 3, | 864 whio_epfs_index_label,
831 /** Inode table entry. 865 /** Inode table entry.
832 866
833 fs->offsets[whio_epfs_index_inode] == offset into EFS of inode table. 867 fs->offsets[whio_epfs_index_inode] == offset into EFS of inode table.
834 868
835 fs->sizes[whio_epfs_index_inode] == encoded size of whio_epfs_inode. 869 fs->sizes[whio_epfs_index_inode] == encoded size of whio_epfs_inode.
836 */ 870 */
837 whio_epfs_index_inode = 4, | 871 whio_epfs_index_inode,
838 /** Block table entry. 872 /** Block table entry.
839 873
840 fs->offsets[whio_epfs_index_blockMeta] == offset into EFS of blocks table. 874 fs->offsets[whio_epfs_index_blockMeta] == offset into EFS of blocks table.
841 875
842 fs->sizes[whio_epfs_index_blockMeta] == encoded size of whio_epfs_block. 876 fs->sizes[whio_epfs_index_blockMeta] == encoded size of whio_epfs_block.
843 */ 877 */
844 whio_epfs_index_blockMeta = 5, | 878 whio_epfs_index_blockMeta,
845 /** MUST be the last entry in this enum. */ 879 /** MUST be the last entry in this enum. */
846 whio_epfs_index_count = 6 880 whio_epfs_index_count = 6
847 }; 881 };
848 /** 882 /**
849 The whio_epfs_sizeof enum defines the on-storage sizes of 883 The whio_epfs_sizeof enum defines the on-storage sizes of
35 hidden lines
885 /** 919 /**
886 Encoded size of whio_epfs_hints. 920 Encoded size of whio_epfs_hints.
887 */ 921 */
888 whio_epfs_sizeof_hints = 1/*tag byte*/ 922 whio_epfs_sizeof_hints = 1/*tag byte*/
889 + (5 * whio_epfs_sizeof_id) /* whio_epfs::whio_epfs_hints::( 923 + (5 * whio_epfs_sizeof_id) /* whio_epfs::whio_epfs_hints::(
> 924 maxEverUsedBlock,
890 maxEverUsedInode, 925 maxEverUsedInode,
891 freeInodeList, 926 freeInodeList,
892 freeBlockList, 927 freeBlockList,
893 maxEverUsedBlock, <
894 blockCount) */ 928 blockCount) */
895 , 929 ,
896 /** 930 /**
897 Encoded size of whio_epfs_inode. 931 Encoded size of whio_epfs_inode.
898 */ 932 */
207 hidden lines
1106 whio_impl_data impl; 1140 whio_impl_data impl;
1107 }; 1141 };
1108 /** Convenience typedef. */ 1142 /** Convenience typedef. */
1109 typedef struct whio_epfs_namer whio_epfs_namer; 1143 typedef struct whio_epfs_namer whio_epfs_namer;
1110 /** @def whio_epfs_namer_empty_m 1144 /** @def whio_epfs_namer_empty_m
> 1145
1111 Empty initialization whio_epfs_namer object. 1146 Empty initialization whio_epfs_namer object.
1112 */ 1147 */
1113 #define whio_epfs_namer_empty_m {NULL,whio_impl_data_empty_m} 1148 #define whio_epfs_namer_empty_m {NULL,whio_impl_data_empty_m}
1114 /** Empty initialization whio_epfs_namer object. */ 1149 /** Empty initialization whio_epfs_namer object. */
1115 extern const whio_epfs_namer whio_epfs_namer_empty; 1150 extern const whio_epfs_namer whio_epfs_namer_empty;
103 hidden lines
1219 */ 1254 */
1220 whio_epfs_id_t freeInodeList; 1255 whio_epfs_id_t freeInodeList;
1221 1256
1222 /** 1257 /**
1223 ID of the first free block, or 0 if there are none. 1258 ID of the first free block, or 0 if there are none.
> 1259
> 1260 Each block contains a member
> 1261 (whio_epfs_block::nextFree) which points to the next
> 1262 free item in the chain, or 0 if the block itself is
> 1263 used (and no longer in the free chain).
1224 */ 1264 */
1225 whio_epfs_id_t freeBlockList; 1265 whio_epfs_id_t freeBlockList;
1226 /** 1266 /**
1227 The highest block ID which has been added to the EFS. 1267 The highest block ID which has been added to the EFS.
1228 */ 1268 */
39 hidden lines
1268 /** Memory reserved for use as an allocation source. */ 1308 /** Memory reserved for use as an allocation source. */
1269 unsigned char * mem; 1309 unsigned char * mem;
1270 /** Length of mem, in bytes. */ 1310 /** Length of mem, in bytes. */
1271 whio_size_t size; 1311 whio_size_t size;
1272 } pool; 1312 } pool;
1273 /** Will replace the pool meember. */ | 1313 /**
| 1314 Allocator used for de/re/allocating EFS-internal
| 1315 memory. Will someday completely replace the pool member.
| 1316 */
1274 whio_allocator alloc; 1317 whio_allocator alloc;
1275 /** Not yet used. */ 1318 /** Not yet used. */
1276 struct whio_epfs_namer_ 1319 struct whio_epfs_namer_
1277 { 1320 {
1278 whio_epfs_namer * n; 1321 whio_epfs_namer * n;
264 hidden lines
1543 to *fs, else fs is assumed to point to a clean, 1586 to *fs, else fs is assumed to point to a clean,
1544 client-allocated whio_epfs object. 1587 client-allocated whio_epfs object.
1545 1588
1546 How sopt is interpreted, and possibly changed, is described in 1589 How sopt is interpreted, and possibly changed, is described in
1547 whio_epfs_openfs(), so please see that function for how to 1590 whio_epfs_openfs(), so please see that function for how to
1548 intialize and populate it. | 1591 intialize and populate it, and regarding ownership of objects
| 1592 stored in the sopt parameter.
1549 1593
1550 The fsopt parameter describes the basic parameters of the 1594 The fsopt parameter describes the basic parameters of the
1551 filesystem. If certain ranges or combinations are violated 1595 filesystem. If certain ranges or combinations are violated
1552 then whio_rc.RangeError may be returned. 1596 then whio_rc.RangeError may be returned.
1553 1597
2 hidden lines
1556 must eventually free the returned object using 1600 must eventually free the returned object using
1557 whio_epfs_finalize(). If he passed in his own object then it 1601 whio_epfs_finalize(). If he passed in his own object then it
1558 must be cleaned up as appropriate (whio_epfs_finalize() for 1602 must be cleaned up as appropriate (whio_epfs_finalize() for
1559 heap-allocated and whio_epfs_close() for stack-allocated). 1603 heap-allocated and whio_epfs_close() for stack-allocated).
1560 1604
> 1605 Example:
> 1606
> 1607 @code
> 1608 // Set up general fs options:
> 1609 whio_epfs_fsopt fo = whio_epfs_fsopt_empty;
> 1610 fo.maxBlocks = 1000; // 0=grow on demand
> 1611 fo.blockSize = 1024 * 16;
> 1612 fo.inodeCount = 250;
> 1613
> 1614 // Open/mkfs-specific options, including the storage device:
> 1615 whio_epfs_setup_opt so = whio_epfs_setup_opt_empty;
> 1616 so.storage.dev = whio_dev_for_filename("my.epfs","w+");
> 1617 so.storage.takeDevOnSuccess = whio_dev_for_filename("my.epfs","w+");
> 1618
> 1619 // Create the fs:
> 1620 whio_efps * fs = NULL;
> 1621 rc = whio_epfs_mkfs( &fs, &so, &fo );
> 1622 if( rc ) { //error
> 1623 // We need to clean up the storage device:
> 1624 so.storage.dev->api->finalize( so.storage.dev );
> 1625 return ...;
> 1626 }
> 1627 ... use fs ...
> 1628 whio_epfs_finalize( fs );
> 1629 @endcode
> 1630
> 1631 See also the convenience function whio_epfs_mkfs_file(), which
> 1632 takes a slightly higher-level set of parameters.
> 1633
1561 @see whio_epfs_mkfs2() 1634 @see whio_epfs_mkfs2()
1562 @see whio_epfs_openfs() 1635 @see whio_epfs_openfs()
> 1636 @see whio_epfs_mkfs_file()
1563 */ 1637 */
1564 int whio_epfs_mkfs( whio_epfs ** fs, whio_epfs_setup_opt * sopt, whio_epfs_fsopt const * fsopt ); 1638 int whio_epfs_mkfs( whio_epfs ** fs, whio_epfs_setup_opt * sopt, whio_epfs_fsopt const * fsopt );
1565 1639
1566 /** 1640 /**
1567 This is a short-hand form of whio_epfs_mkfs() which uses 1641 This is a short-hand form of whio_epfs_mkfs() which uses
87 hidden lines
1655 1729
1656 1730
1657 Lifetime of the opt object: 1731 Lifetime of the opt object:
1658 1732
1659 After this function returns, fs does not directly point to any 1733 After this function returns, fs does not directly point to any
1660 which is still set in the opt object (remember that pointers in | 1734 memory which is still set in the opt object (remember that
1661 the opt object may be modified by this function - see above), and | 1735 pointers in the opt object may be modified by this function -
1662 the opt object need not outlive fs. | 1736 see above), and the opt object need not outlive fs.
1663 1737
1664 1738
1665 Return codes: 1739 Return codes:
1666 1740
1667 Returns 0 on success. On error, the fs will be internally 1741 Returns 0 on success. On error, the fs will be internally
89 hidden lines
1757 If writeMode is true then the device is opened in read/write mode, 1831 If writeMode is true then the device is opened in read/write mode,
1758 otherwise it is opened in read-only mode. 1832 otherwise it is opened in read-only mode.
1759 1833
1760 Returns 0 on success. On error non-zero is returned and *fs is 1834 Returns 0 on success. On error non-zero is returned and *fs is
1761 never modified. 1835 never modified.
> 1836
> 1837
> 1838 @see whio_epfs_mkfs_file()
> 1839 @see whio_epfs_mkfs()
> 1840 @see whio_epfs_open()
1762 */ 1841 */
1763 int whio_epfs_openfs_file( whio_epfs ** fs, char const * filename, bool writeMode ); 1842 int whio_epfs_openfs_file( whio_epfs ** fs, char const * filename, bool writeMode );
> 1843
1764 /** 1844 /**
1765 Tries to initialize a file as an EPFS container. 1845 Tries to initialize a file as an EPFS container.
1766 1846
1767 None of the parameters may be 0. 1847 None of the parameters may be 0.
1768 1848
9 hidden lines
1778 to make a new EPFS filesystem. 1858 to make a new EPFS filesystem.
1779 1859
1780 If creation of the fs fails then the file may be left in an 1860 If creation of the fs fails then the file may be left in an
1781 inconsistent state and should be removed (or analysed, or 1861 inconsistent state and should be removed (or analysed, or
1782 whatever) by the client. 1862 whatever) by the client.
> 1863
> 1864 @see whio_epfs_mkfs()
> 1865 @see whio_epfs_open()
> 1866 @see whio_epfs_openfs_file()
1783 */ 1867 */
1784 int whio_epfs_mkfs_file( whio_epfs ** fs, 1868 int whio_epfs_mkfs_file( whio_epfs ** fs,
1785 whio_epfs_fsopt const * fsopt, 1869 whio_epfs_fsopt const * fsopt,
1786 char const * filename, 1870 char const * filename,
1787 bool allowOverwrite ); 1871 bool allowOverwrite );
2 hidden lines
1790 /** 1874 /**
1791 Please read all of these docs before using this function... 1875 Please read all of these docs before using this function...
1792 1876
1793 First: HIGHLY EXPERIMENTAL! It normally seems to work, but once 1877 First: HIGHLY EXPERIMENTAL! It normally seems to work, but once
1794 in a while i'm getting memory corruption in objects allocated 1878 in a while i'm getting memory corruption in objects allocated
1795 through the custom allocator, normally via the block array | 1879 through the custom allocator, seemingly via the block array
1796 reallocations (which hints at a bug in the realloc impl). 1880 reallocations (which hints at a bug in the realloc impl).
1797 1881
1798 Until the above warning is gone, please do not use this 1882 Until the above warning is gone, please do not use this
1799 function or the related functionality built on top of it 1883 function or the related functionality built on top of it
1800 (e.g. whio_epfs_setup_opt::memory). 1884 (e.g. whio_epfs_setup_opt::memory).
73 hidden lines
1874 The following list contains the ONLY functions which may 1958 The following list contains the ONLY functions which may
1875 legally be called _before_ whio_epfs_mempool_setup(). All 1959 legally be called _before_ whio_epfs_mempool_setup(). All
1876 others may indirectly induce undefined behaviour if 1960 others may indirectly induce undefined behaviour if
1877 whio_epfs_mempool_setup() is called after they are called: 1961 whio_epfs_mempool_setup() is called after they are called:
1878 1962
1879 - whio_epfs_mkfs2() | 1963 - whio_epfs_mkfs() (not whio_epfs_mkfs2())
1880 - whio_epfs_openfs2() | 1964 - whio_epfs_openfs() (not whio_epfs_openfs2())
1881 1965
1882 To set up a memory pool at the time the fs is initialized, as 1966 To set up a memory pool at the time the fs is initialized, as
1883 opposed to afterwards, use whio_epfs_mkfs() or 1967 opposed to afterwards, use whio_epfs_mkfs() or
1884 whio_epfs_openfs(). Doing so removes any concerns about legal 1968 whio_epfs_openfs(). Doing so removes any concerns about legal
1885 call ordering. 1969 call ordering.
1886 1970
1887 If you actually read this far, you have my blessing to use this 1971 If you actually read this far, you have my blessing to use this
1888 function. If you skipped to the end, please go back and read 1972 function. If you skipped to the end, please go back and read
1889 these docs before using it. | 1973 these docs before using it. And then if you see weird behaviour
| 1974 please try not using this, to be sure that the allocator is not
| 1975 the problem.
1890 */ 1976 */
1891 int whio_epfs_mempool_setup( whio_epfs * fs, void * mem, whio_size_t size, bool fallback ); 1977 int whio_epfs_mempool_setup( whio_epfs * fs, void * mem, whio_size_t size, bool fallback );
1892 1978
1893 /** 1979 /**
1894 A type for collecting certain metrics from a whio_epfs memory 1980 A type for collecting certain metrics from a whio_epfs memory
79 hidden lines
1974 2060
1975 If fs was not created with whio_epfs_alloc() then results 2061 If fs was not created with whio_epfs_alloc() then results
1976 are technically undefined. That said, the implementation 2062 are technically undefined. That said, the implementation
1977 tries to detect if fs was allocated via whio_epfs_alloc(), 2063 tries to detect if fs was allocated via whio_epfs_alloc(),
1978 and does not actually free it if it was not (which may lead 2064 and does not actually free it if it was not (which may lead
1979 to a leak). | 2065 to a leak, but won't immediately crash on you).
1980 2066
1981 Returns 0 on success. The only reported error condition is when 2067 Returns 0 on success. The only reported error condition is when
1982 !fs. However, if fs was not allocated by whio_epfs_alloc(), this 2068 !fs. However, if fs was not allocated by whio_epfs_alloc(), this
1983 routine will effectively close it, but also _not_ deallocate it, | 2069 routine will effectively close it, but will _not_ deallocate it,
1984 and will report success. 2070 and will report success.
1985 */ 2071 */
1986 int whio_epfs_finalize( whio_epfs *fs ); 2072 int whio_epfs_finalize( whio_epfs *fs );
1987 2073
1988 /** Returns true only if fs is not null and is opened for | 2074 /** Returns true only if fs is not null and is opened in
1989 read/write. */ | 2075 read/write mode.
| 2076 */
1990 bool whio_epfs_is_rw( whio_epfs const * const fs ); 2077 bool whio_epfs_is_rw( whio_epfs const * const fs );
1991 2078
1992 /** 2079 /**
1993 If fs is opened for read/write then this flushes the storage 2080 If fs is opened for read/write then this flushes the storage
1994 and returns a storage-dependent non-zero value on error. If fs 2081 and returns a storage-dependent non-zero value on error. If fs
1995 is read-only then whio_rc.AccessError is returned. If !fs then 2082 is read-only then whio_rc.AccessError is returned. If !fs then
1996 whio_rc.ArgError is returned. 2083 whio_rc.ArgError is returned.
1997 2084
1998 This routine is used to ensure that any persistent fs-internal | 2085 The difference between calling this and, say, the flush()
1999 metadata is flushed to storage. Do not call it too often, | 2086 member of a device returned from whio_epfs_dev_open(), is that
2000 though, as it can be relatively slow. | 2087 this flushes the underlying storage of the EFS container,
| 2088 whereas the latter only ensures that the current inode state is
| 2089 written to that storage (but not necessarily flushed).
| 2090
| 2091 Do not call it too often, as it can be relatively slow.
| 2092
| 2093 It is not normally necessary for client code to call this,
| 2094 but internals which muck with persistent
2001 2095
2002 Returns 0 on success. 2096 Returns 0 on success.
2003 */ 2097 */
2004 int whio_epfs_flush( whio_epfs * fs ); 2098 int whio_epfs_flush( whio_epfs * fs );
2005 2099
11 hidden lines
2017 whio_epfs_id_t whio_epfs_block_count( whio_epfs const * fs ); 2111 whio_epfs_id_t whio_epfs_block_count( whio_epfs const * fs );
2018 2112
2019 /** 2113 /**
2020 Opens the given inode for random access with the given 2114 Opens the given inode for random access with the given
2021 read/write mode. If inodeID is 0 then mode must include 2115 read/write mode. If inodeID is 0 then mode must include
2022 WHIO_EPFS_MODE_FLAG_CREATE, and the next available (free/unused) | 2116 WHIO_EPFS_MODE_FLAG_CREATE and the next available (free/unused)
2023 inode will be opened for read/write access. 2117 inode will be opened for read/write access.
2024 2118
2025 dev must be non-null and *dev must either point to null or 2119 dev must be non-null and *dev must either point to null or
2026 an empty-initialized objects. On success, the created whio_dev object will 2120 an empty-initialized objects. On success, the created whio_dev object will
2027 be stored there. It must be closed or finalized as described in 2121 be stored there. It must be closed or finalized as described in
17 hidden lines
2045 "create" because of its logical similarity to the O_CREAT flag 2139 "create" because of its logical similarity to the O_CREAT flag
2046 for the open(2) system call. In this mode, if id==0 and no 2140 for the open(2) system call. In this mode, if id==0 and no
2047 unused inodes are found then this function will fail 2141 unused inodes are found then this function will fail
2048 with whio_rc.DeviceFullError. 2142 with whio_rc.DeviceFullError.
2049 2143
2050 - The WHIO_EPFS_MODE_FLAG_RWC flag is equivalent to | 2144 - The WHIO_EPFS_MODE_RWC flag is equivalent to
2051 (WHIO_EPFS_MODE_FLAG_RW | WHIO_EPFS_MODE_FLAG_CREATE). | 2145 (WHIO_EPFS_MODE_RW | WHIO_EPFS_MODE_FLAG_CREATE).
2052 2146
2053 - WHIO_EPFS_MODE_FLAG_TRUNCATE can be OR'd together with 2147 - WHIO_EPFS_MODE_FLAG_TRUNCATE can be OR'd together with
2054 WHIO_EPFS_MODE_WRITE to signify that the device should be 2148 WHIO_EPFS_MODE_WRITE to signify that the device should be
2055 truncated to 0 bytes after it is opened. Under very tight 2149 truncated to 0 bytes after it is opened. Under very tight
2056 memory conditions truncation can actually fail (because it must 2150 memory conditions truncation can actually fail (because it must
34 hidden lines
2091 allocation method). 2185 allocation method).
2092 2186
2093 Closing/finalizing dev after fs is closed will lead to 2187 Closing/finalizing dev after fs is closed will lead to
2094 undefined behaviour. 2188 undefined behaviour.
2095 2189
> 2190 Example:
> 2191
> 2192 @code
> 2193 whio_dev * d = NULL
> 2194 int rc = whio_epfs_dev_open( myfs, &d, 0, WHIO_EPFS_MODE_RWC );
> 2195 if( rc ) { ... error ... }
> 2196 else {
> 2197 whio_epfs_id_t id = whio_epfs_dev_inode_id(d);
> 2198 // ... use device, then destroy it ...
> 2199 d->api->finalize(d);
> 2200 }
> 2201 @endcode
> 2202
2096 whio_dev interface notes: 2203 whio_dev interface notes:
2097 2204
2098 Peculiarities of the returned whio_dev device vis-a-vis the 2205 Peculiarities of the returned whio_dev device vis-a-vis the
2099 whio_dev standard interface: 2206 whio_dev standard interface:
2100 2207
14 hidden lines
2115 fetch it.) 2222 fetch it.)
2116 2223
2117 - Calling whio_dev_ioctl(dev,...) with a second argument of 2224 - Calling whio_dev_ioctl(dev,...) with a second argument of
2118 whio_epfs_ioctl_INODE_PTR and a third argument of type 2225 whio_epfs_ioctl_INODE_PTR and a third argument of type
2119 (whio_epfs_inode**) will set the 3rd argument to the pointer 2226 (whio_epfs_inode**) will set the 3rd argument to the pointer
2120 to the inode associated with the device. (Or use | 2227 to the inode associated with the device. (Or use whio_epfs_dev_inode()
| 2228 to fetch it.)
2121 2229
2122 - Calling whio_dev_ioctl(dev,...) with a second argument of 2230 - Calling whio_dev_ioctl(dev,...) with a second argument of
2123 whio_epfs_ioctl_HANDLE_PTR and a third argument of type 2231 whio_epfs_ioctl_HANDLE_PTR and a third argument of type
2124 (whio_epfs_HANDLE**) will set the 3rd argument to the pointer 2232 (whio_epfs_HANDLE**) will set the 3rd argument to the pointer
2125 to the handle associated with the device. NOTE that handle->inode | 2233 to the handle associated with the device. NOTE that
2126 might not be the real inode entry used by the handle (this | 2234 handle->inode might not be the real inode entry used by the
2127 happens when a handle is opened multiple times), and thus the | 2235 handle (this happens when a handle is opened multiple times),
2128 whio_epfs_ioctl_INODE_PTR ioctl should be used to fetch the | 2236 and thus the whio_epfs_dev_inode() should be used to fetch the
2129 inode, instead of using handle->inode directly. | 2237 inode, if needed, instead of using handle->inode directly.
2130 2238
2131 - The dev->client member is not used by this API, and can be 2239 - The dev->client member is not used by this API, and can be
2132 used by the client as described in the documentation for 2240 used by the client as described in the documentation for
2133 whio_dev::client. 2241 whio_dev::client.
2134 2242
2135 OPENING AN INODE MULTIPLE TIMES: | 2243 Having said all of that about getting a pointer to the device's
| 2244 internal inode and handle data... don't do it unless you are
| 2245 writing a tool for use with this library and you are aware that
| 2246 any incorrect fiddling of the inode/handle objects can corrupt
| 2247 the EFS, leak memory, or otherwise cause Undefined Behaviour.
| 2248
| 2249
| 2250 Opening an inode multiple times:
2136 2251
2137 If a given inode is opened multiple times, all opened 2252 If a given inode is opened multiple times, all opened
2138 references to it share the same underlying inode object and 2253 references to it share the same underlying inode object and
2139 block chain but each has their own logical file position cursor 2254 block chain but each has their own logical file position cursor
2140 and access mode. Thus when updating a given inode, read-only 2255 and access mode. Thus when updating a given inode, read-only
9 hidden lines
2150 closed. This is required so that the inode instance which is 2265 closed. This is required so that the inode instance which is
2151 shared amongst linked handles does not change memory locations 2266 shared amongst linked handles does not change memory locations
2152 at runtime (which would invalidate pointers to it). 2267 at runtime (which would invalidate pointers to it).
2153 2268
2154 By the way, no inode can be opened more than 2269 By the way, no inode can be opened more than
2155 (2^(sizeof(whio_epfs_handle::openCount))) times | 2270 (2^(8*sizeof(whio_epfs_handle::openCount))) times
2156 concurrently. Trying to open it more times than that will fail 2271 concurrently. Trying to open it more times than that will fail
2157 with error code whio_rc.RangeError. 2272 with error code whio_rc.RangeError.
2158 2273
2159 MISFEATURES: | 2274 Misfeatures:
2160 2275
2161 - If *dev is closed after fs is closed, there is a potential 2276 - If *dev is closed after fs is closed, there is a potential
2162 crash condition. Fixing this is on the TODO list but requires 2277 crash condition. Fixing this is on the TODO list but requires
2163 more infrastructure in the whio_epfs class to maintain the 2278 more infrastructure in the whio_epfs class to maintain the
2164 object links. | 2279 object links. In any case, closing the device after fs is
| 2280 closed is in Direct Violation of the API documentation, so i'm
| 2281 in no hurry to add that infrastructure :). (That said, in
| 2282 script-engine binding cases lifetimes can prove more
| 2283 problematic, so this will likely eventually be fixed.)
2165 */ 2284 */
2166 int whio_epfs_dev_open( whio_epfs * fs, whio_dev ** dev, whio_epfs_id_t inodeID, enum whio_epfs_iomodes mode ); 2285 int whio_epfs_dev_open( whio_epfs * fs, whio_dev ** dev, whio_epfs_id_t inodeID, enum whio_epfs_iomodes mode );
2167 2286
2168 /** 2287 /**
2169 If d was initialized via whio_epfs_dev_open() then its associated 2288 If d was initialized via whio_epfs_dev_open() then its associated
10 hidden lines
2180 If dev was initialized for read/write access via 2299 If dev was initialized for read/write access via
2181 whio_epfs_open_dev() then its associated inode is returned. In 2300 whio_epfs_open_dev() then its associated inode is returned. In
2182 any other case, NULL is returned. 2301 any other case, NULL is returned.
2183 2302
2184 It is up to the caller to behave responsibly with the inode's 2303 It is up to the caller to behave responsibly with the inode's
2185 data. Changing certain fields (especially | 2304 data. Changing certain fields (especially whio_epfs_inode::size
2186 whio_epfs_inode::firstBlock) could lead to data loss or EFS | 2305 or whio_epfs_inode::firstBlock) could lead to data loss or EFS
2187 corruption. 2306 corruption.
2188 2307
2189 The returned inode is owned by the containing EPFS engine, and 2308 The returned inode is owned by the containing EPFS engine, and
2190 must not be deallocated nor dereferenced after dev has been 2309 must not be deallocated nor dereferenced after dev has been
2191 closed. 2310 closed.
354 hidden lines
2546 #ifdef __cplusplus 2665 #ifdef __cplusplus
2547 } /* extern "C" */ 2666 } /* extern "C" */
2548 #endif 2667 #endif
2549 2668
2550 #endif /* WANDERINGHORSE_NET_WHIO_EPFS_H_INCLUDED */ 2669 #endif /* WANDERINGHORSE_NET_WHIO_EPFS_H_INCLUDED */

Changes to include/wh/whio/whio_epfs_config.h

Old (45c31476abd10faa) New (35a2cdeb0af140f5)
1 #ifndef WANDERINGHORSE_NET_WHIO_EPFS_CONFIG_H_INCLUDED 1 #ifndef WANDERINGHORSE_NET_WHIO_EPFS_CONFIG_H_INCLUDED
2 #define WANDERINGHORSE_NET_WHIO_EPFS_CONFIG_H_INCLUDED 2 #define WANDERINGHORSE_NET_WHIO_EPFS_CONFIG_H_INCLUDED
3 /** @file whio_epfs_config.h 3 /** @file whio_epfs_config.h
4 4
5 This file contains the compile-time-configurable parts of whio_epfs. 5 This file contains the compile-time-configurable parts of whio_epfs.
143 hidden lines
149 149
150 @see to WHIO_EPFS_ID_T_PFMT 150 @see to WHIO_EPFS_ID_T_PFMT
151 @see to WHIO_EPFS_ID_T_SFMT 151 @see to WHIO_EPFS_ID_T_SFMT
152 */ 152 */
153 153
> 154 /** @typedef SOME_UNSIGNED_INTEGER_TYPE whio_epfs_id_t
> 155
> 156 whio_epfs_id_t is a fixed-sized unsigned integer type
> 157 whose size is WHIO_EPFS_ID_T_BITS/8 bytes.
> 158 */
154 #if WHIO_EPFS_ID_T_BITS == 8 159 #if WHIO_EPFS_ID_T_BITS == 8
155 /* 160 /*
156 For very, very limited filesystems. There's lots of room for 161 For very, very limited filesystems. There's lots of room for
157 overflows here! Completely untested! 162 overflows here! Completely untested!
158 */ 163 */
32 hidden lines
191 #ifdef __cplusplus 196 #ifdef __cplusplus
192 } /* extern "C" */ 197 } /* extern "C" */
193 #endif 198 #endif
194 199
195 #endif /* WANDERINGHORSE_NET_WHIO_EPFS_CONFIG_H_INCLUDED */ 200 #endif /* WANDERINGHORSE_NET_WHIO_EPFS_CONFIG_H_INCLUDED */

Changes to pfs/handle.c

Old (3ea415da23ddacdc) New (b2007569278a8d76)
1 //#define WHIO_DEBUG_ENABLED 1 1 //#define WHIO_DEBUG_ENABLED 1
2 #include "whio_epfs_internal.h" 2 #include "whio_epfs_internal.h"
3 #include <stdlib.h> // malloc() and friends. 3 #include <stdlib.h> // malloc() and friends.
4 #include <string.h> // memmove() and friends 4 #include <string.h> // memmove() and friends
5 #include <assert.h> 5 #include <assert.h>
284 hidden lines
290 { /* fs is r/o but caller asked for r/w. */ 290 { /* fs is r/o but caller asked for r/w. */
291 return whio_rc.AccessError; 291 return whio_rc.AccessError;
292 } 292 }
293 whio_epfs_inode oni = whio_epfs_inode_empty; 293 whio_epfs_inode oni = whio_epfs_inode_empty;
294 int rc = 0; 294 int rc = 0;
295 whio_epfs_handle * h = 0; | 295 whio_epfs_handle * h = NULL;
296 whio_epfs_handle * origin = 0; | 296 whio_epfs_handle * origin = NULL;
297 whio_epfs_id_t i = 0; 297 whio_epfs_id_t i = 0;
298 if(id) for( ; i < fs->handles.count; ++i ) 298 if(id) for( ; i < fs->handles.count; ++i )
299 { // FIXME: keep list sorted on inode id to make this faster. 299 { // FIXME: keep list sorted on inode id to make this faster.
300 h = fs->handles.list[i]; 300 h = fs->handles.list[i];
301 #if 0 301 #if 0
102 hidden lines
404 ? whio_epfs_inode_empty /* we don't want that inode to ever be used. */ 404 ? whio_epfs_inode_empty /* we don't want that inode to ever be used. */
405 : oni; 405 : oni;
406 h->pfs = fs; 406 h->pfs = fs;
407 whio_epfs_handle_list_append( &fs->handles, h ); 407 whio_epfs_handle_list_append( &fs->handles, h );
408 if( ownsHandle ) *tgt = h; 408 if( ownsHandle ) *tgt = h;
> 409 #if 0
> 410 whio_epfs_flush( fs )
> 411 /*Ensure recent free-list changes are written, but adds a
> 412 noticable about to runtime to some uses cases (just over 30%
> 413 in my quick tests).
> 414 */
> 415 ;
> 416 #endif
409 return whio_rc.OK; 417 return whio_rc.OK;
410 } 418 }
411 419
412 int whio_epfs_handle_close( whio_epfs_handle * h ) 420 int whio_epfs_handle_close( whio_epfs_handle * h )
413 { 421 {
29 hidden lines
443 if( ! h->openCount ) 451 if( ! h->openCount )
444 { 452 {
445 #if XXX 453 #if XXX
446 WHIO_DEBUG("Freeing handle #%"WHIO_EPFS_ID_T_PFMT" @%p\n", origin->inode.id, (void const *)h ); 454 WHIO_DEBUG("Freeing handle #%"WHIO_EPFS_ID_T_PFMT" @%p\n", origin->inode.id, (void const *)h );
447 #endif 455 #endif
448 whio_epfs_handle_search( &h->pfs->handles, h, true ); | 456 //whio_epfs * fs = origin->pfs;
| 457 whio_epfs_handle_search( &origin->pfs->handles, h, true );
449 whio_epfs_handle_free(h); 458 whio_epfs_handle_free(h);
> 459 #if 0
> 460 whio_epfs_flush( fs ) /* ensure recent free-list changes are written.*/;
> 461 #endif
450 rc = whio_rc.OK; 462 rc = whio_rc.OK;
451 } 463 }
452 if( (origin != h) ) 464 if( (origin != h) )
453 { 465 {
454 { 466 {
6 hidden lines
461 #undef XXX 473 #undef XXX
462 return rc; 474 return rc;
463 } 475 }
464 476
465 #undef TRY_SORTED_HANDLE_LIST 477 #undef TRY_SORTED_HANDLE_LIST

Changes to pfs/inode.c

Old (0032a6a00366197d) New (307b61ef63d19816)
1 /************************************************************************ 1 /************************************************************************
2 This file contains most of the inode-specific whio_epfs routines. 2 This file contains most of the inode-specific whio_epfs routines.
3 3
4 Author: Stephan Beal (http://wanderinghorse.net/home/stephan/) 4 Author: Stephan Beal (http://wanderinghorse.net/home/stephan/)
5 5
274 hidden lines
280 { 280 {
281 fs->hints.freeInodeList = ino.nextFree; 281 fs->hints.freeInodeList = ino.nextFree;
282 } 282 }
283 id = ino.nextFree; 283 id = ino.nextFree;
284 continue; 284 continue;
> 285 }
> 286 else
> 287 {
> 288 return whio_rc.ConsistencyError/*???*/;
285 } 289 }
286 } 290 }
287 break; 291 break;
288 } 292 }
289 if( ! id ) 293 if( ! id )
242 hidden lines
532 } 536 }
533 return rc; 537 return rc;
534 } 538 }
535 539
536 #undef MARKER 540 #undef MARKER

Changes to pfs/ls.c

Old (c9e8d146738a5388) New (3108401899ca70b3)
1 /************************************************************************ 1 /************************************************************************
2 An "ls"-like program for use with whio_epfs. 2 An "ls"-like program for use with whio_epfs.
3 3
4 Author: Stephan Beal (http://wanderinghorse.net/home/stephan/) 4 Author: Stephan Beal (http://wanderinghorse.net/home/stephan/)
5 5
89 hidden lines
95 0U /*blockCount*/, 95 0U /*blockCount*/,
96 0U /*inodeCount*/ 96 0U /*inodeCount*/
97 }; 97 };
98 98
99 /** 99 /**
100 Appends f to AppLs.funcs. | 100 Appends f to AppLs.funcs. exit()s on error (called
| 101 AppLs_MAX_COMMANDS times).
101 */ 102 */
102 static int AppLs_push_command( ls_command_f f ) 103 static int AppLs_push_command( ls_command_f f )
103 { 104 {
104 assert( AppLs.funcs.count < AppLs_MAX_COMMANDS ); | 105 if( AppLs.funcs.count >= AppLs_MAX_COMMANDS )
| 106 {
| 107 APPERR("Internal range error: too many (%u) operations!\n",(unsigned int) AppLs.funcs.count);
| 108 exit(whio_rc.InternalError);
| 109 }
105 AppLs.funcs.f[AppLs.funcs.count++] = f; 110 AppLs.funcs.f[AppLs.funcs.count++] = f;
106 return 0; 111 return 0;
107 } 112 }
108 113
109 /** 114 /**
132 hidden lines
242 int rc = whio_epfs_foreach_inode( EPFSApp.fs, NULL, NULL, inode_foreach_ls, &info ); 247 int rc = whio_epfs_foreach_inode( EPFSApp.fs, NULL, NULL, inode_foreach_ls, &info );
243 248
244 249
245 if( ! AppLs.oneMode ) 250 if( ! AppLs.oneMode )
246 { 251 {
247 printf("\nTotal: %"WHIO_EPFS_ID_T_PFMT" of %"WHIO_EPFS_ID_T_PFMT" inodes take up " | 252 printf("\nTotals: %"WHIO_EPFS_ID_T_PFMT" of %"WHIO_EPFS_ID_T_PFMT" inodes take up "
248 "%"WHIO_SIZE_T_PFMT" bytes", 253 "%"WHIO_SIZE_T_PFMT" bytes",
249 info.inodeCount, o->inodeCount, 254 info.inodeCount, o->inodeCount,
250 info.totalSize 255 info.totalSize
251 ); 256 );
252 if( AppLs.showBlocks ) 257 if( AppLs.showBlocks )
253 { 258 {
254 printf(" in %"WHIO_EPFS_ID_T_PFMT" blocks", 259 printf(" in %"WHIO_EPFS_ID_T_PFMT" blocks",
255 info.blockCount ); 260 info.blockCount );
256 } 261 }
257 puts("."); | 262 puts(".\n");
258 } 263 }
259 264
260 if( rc ) 265 if( rc )
261 { 266 {
262 APPERR("Got error code %d while looping over inodes!\n", rc ); 267 APPERR("Got error code %d while looping over inodes!\n", rc );
15 hidden lines
278 283
279 static int ls_dump_freelists_command() 284 static int ls_dump_freelists_command()
280 { 285 {
281 int rc = 0; 286 int rc = 0;
282 int i = 0; 287 int i = 0;
283 int span = 4; | 288 const int span = 4;
284 { 289 {
285 puts("Data block free-list: (Block ID->Next free ID)\n"); | 290 puts("Data block free-list: (blockID->nextFreeBlock)\n");
286 whio_epfs_id_t f = EPFSApp.fs->hints.freeBlockList; 291 whio_epfs_id_t f = EPFSApp.fs->hints.freeBlockList;
287 if( !f ) 292 if( !f )
288 { 293 {
289 puts("No explicit unused blocks."); | 294 if( !EPFSApp.fs->fsopt.maxBlocks )
| 295 {
| 296 puts("\tNo free blocks, but this EFS can grow on demand.");
| 297 }
| 298 else
| 299 {
| 300 puts("\tNo unused blocks - this EFS is full.");
| 301 }
290 } 302 }
291 else 303 else
292 { 304 {
> 305 unsigned int count = 0;
293 putchar('\t'); 306 putchar('\t');
294 while( f ) 307 while( f )
295 { 308 {
> 309 ++count;
296 whio_epfs_block bl = whio_epfs_block_empty; 310 whio_epfs_block bl = whio_epfs_block_empty;
297 rc = whio_epfs_block_read( EPFSApp.fs, f, &bl ); 311 rc = whio_epfs_block_read( EPFSApp.fs, f, &bl );
298 if( rc ) 312 if( rc )
299 { 313 {
300 APPERR("Error reading block #%"WHIO_EPFS_ID_T_PFMT"!\n",f); 314 APPERR("Error reading block #%"WHIO_EPFS_ID_T_PFMT"!\n",f);
9 hidden lines
310 i = 0; 324 i = 0;
311 printf("\n%s",(f ? "\t" : "")); 325 printf("\n%s",(f ? "\t" : ""));
312 } 326 }
313 else ++i; 327 else ++i;
314 } 328 }
> 329 printf("\n%s\t= %u unused blocks in free-list\n",(i?"\n":""),count);
315 } 330 }
316 putchar('\n'); <
317 } 331 }
318 putchar('\n'); 332 putchar('\n');
319 { 333 {
320 i = 0; 334 i = 0;
321 puts("Inode free-list: (Previous Free<- Inode ID -> Next Free)\n"); | 335 puts("Inode free-list: (previous<-current->next)\n");
322 whio_epfs_id_t f = EPFSApp.fs->hints.freeInodeList; 336 whio_epfs_id_t f = EPFSApp.fs->hints.freeInodeList;
323 if( !f ) 337 if( !f )
324 { 338 {
325 puts("No explicit unused inodes."); | 339 puts("\tNo free inodes.");
326 } 340 }
327 else 341 else
328 { 342 {
> 343 unsigned int count = 0;
329 putchar('\t'); 344 putchar('\t');
330 while( f ) 345 while( f )
331 { 346 {
> 347 ++count;
332 whio_epfs_inode ino = whio_epfs_inode_empty; 348 whio_epfs_inode ino = whio_epfs_inode_empty;
333 rc = whio_epfs_inode_read( EPFSApp.fs, f, &ino ); 349 rc = whio_epfs_inode_read( EPFSApp.fs, f, &ino );
334 if( rc ) 350 if( rc )
335 { 351 {
336 APPERR("Error reading inode #%"WHIO_EPFS_ID_T_PFMT"!\n",f); 352 APPERR("Error reading inode #%"WHIO_EPFS_ID_T_PFMT"!\n",f);
9 hidden lines
346 i = 0; 362 i = 0;
347 printf("\n%s",(f ? "\t" : "")); 363 printf("\n%s",(f ? "\t" : ""));
348 } 364 }
349 else ++i; 365 else ++i;
350 } 366 }
> 367 printf("\n%s\t= %u unused inodes in free-list\n",(i?"\n":""),count);
351 } 368 }
352 putchar('\n'); 369 putchar('\n');
353 } 370 }
354 putchar('\n'); 371 putchar('\n');
355 return 0; 372 return 0;
161 hidden lines
517 } 534 }
518 } 535 }
519 EPFSApp_verbose_error_code( rc ); 536 EPFSApp_verbose_error_code( rc );
520 return rc; 537 return rc;
521 } 538 }

Changes to pfs/pfs.c

Old (17e2c59903607ab5) New (4cc13ad50e092b5c)
1 /************************************************************************ 1 /************************************************************************
2 This is a work-in-progress, porting over parts of the whefs API into 2 This is a work-in-progress, porting over parts of the whefs API into
3 the whio API... 3 the whio API...
4 4
5 Author: Stephan Beal (http://wanderinghorse.net/home/stephan/) 5 Author: Stephan Beal (http://wanderinghorse.net/home/stephan/)
56 hidden lines
62 if(!fs || !fs->dev ) return whio_rc.ArgError; 62 if(!fs || !fs->dev ) return whio_rc.ArgError;
63 else if( !whio_epfs_is_rw(fs) ) return whio_rc.AccessError; 63 else if( !whio_epfs_is_rw(fs) ) return whio_rc.AccessError;
64 else 64 else
65 { 65 {
66 /* FIXME? put this somewhere else? */ 66 /* FIXME? put this somewhere else? */
67 int rc = whio_epfs_hints_write(fs); | 67 int rc =
| 68 #if 1
| 69 0
| 70 #else
| 71 whio_epfs_hints_write(fs)
| 72 #endif
| 73 ;
68 if(! rc ) 74 if(! rc )
69 { 75 {
70 rc = fs->dev->api->flush( fs->dev ); 76 rc = fs->dev->api->flush( fs->dev );
71 } 77 }
72 return rc; 78 return rc;
157 hidden lines
230 int whio_epfs_close( whio_epfs *fs ) 236 int whio_epfs_close( whio_epfs *fs )
231 { 237 {
232 if( !fs ) return whio_rc.ArgError; 238 if( !fs ) return whio_rc.ArgError;
233 if( !fs->err && fs->dev && whio_epfs_is_rw(fs) ) 239 if( !fs->err && fs->dev && whio_epfs_is_rw(fs) )
234 { 240 {
> 241 whio_epfs_hints_write(fs);
235 whio_epfs_flush(fs); 242 whio_epfs_flush(fs);
236 } 243 }
237 if( fs->client.dtor ) 244 if( fs->client.dtor )
238 { 245 {
239 /** 246 /**
432 hidden lines
672 rc = whio_epfs_inode_flush( fs, &ino ); 679 rc = whio_epfs_inode_flush( fs, &ino );
673 } 680 }
674 return rc; 681 return rc;
675 } 682 }
676 683
677 /** | 684 /** @internal
678 Internal code-duplication remover. It assumes fs is fully (but just 685 Internal code-duplication remover. It assumes fs is fully (but just
679 recently) initialized, and that sopt is properly set up. It also 686 recently) initialized, and that sopt is properly set up. It also
680 assumes that initialization of fs is at its end phase, and the fs 687 assumes that initialization of fs is at its end phase, and the fs
681 object has already suceeded through initialization except for these | 688 object has already succeeded through initialization except for these
682 house-keeping bits. 689 house-keeping bits.
683 690
684 If sopt->memory.mem is not null then whio_epfs_mempool_setup() is 691 If sopt->memory.mem is not null then whio_epfs_mempool_setup() is
685 called and on success sopt->memory.mem is set to NULL. If mempool 692 called and on success sopt->memory.mem is set to NULL. If mempool
686 initialization fails, that error code is returned from this 693 initialization fails, that error code is returned from this
687 function. 694 function.
688 695
689 If sopt->storage.takeDevOnSuccess is true then fs has its owns-the-device flag | 696 If sopt->storage.takeDevOnSuccess is true then fs has its
690 set and sopt->storage.dev is set to NULL. It is assumed that fs | 697 owns-the-device flag set and sopt->storage.dev is set to NULL to
| 698 signify to the caller that ownership was taken.
691 699
692 Returns non-zero if any part of setup fails, but these parts are 700 Returns non-zero if any part of setup fails, but these parts are
693 not considered critical, and should not cause init of the fs to 701 not considered critical, and should not cause init of the fs to
694 fail (i.e., ignore the return code, and the client can check sopt's 702 fail (i.e., ignore the return code, and the client can check sopt's
695 state if he NEEDs to know if certain part failed). 703 state if he NEEDs to know if certain part failed).
442 hidden lines
1138 return rc; 1146 return rc;
1139 } 1147 }
1140 1148
1141 1149
1142 #undef MARKER 1150 #undef MARKER

Changes to pfs/whio_epfs_internal.h

Old (59f4708000a3b808) New (cca5cec5d5a97ed8)
1 #ifndef WANDERINGHORSE_NET_WHIO_EPFS_INTERNAL_H_INCLUDED 1 #ifndef WANDERINGHORSE_NET_WHIO_EPFS_INTERNAL_H_INCLUDED
2 #define WANDERINGHORSE_NET_WHIO_EPFS_INTERNAL_H_INCLUDED 2 #define WANDERINGHORSE_NET_WHIO_EPFS_INTERNAL_H_INCLUDED
3 3
4 /** @file whio_epfs_internal.h 4 /** @file whio_epfs_internal.h
5 5
131 hidden lines
137 Tries to read the given inode ID from storage. On success 0 is 137 Tries to read the given inode ID from storage. On success 0 is
138 returned and dest is populated with the fetched values. On 138 returned and dest is populated with the fetched values. On
139 error dest is not modified. 139 error dest is not modified.
140 */ 140 */
141 int whio_epfs_inode_read( whio_epfs * fs, 141 int whio_epfs_inode_read( whio_epfs * fs,
142 whio_epfs_id_t id, | 142 whio_epfs_id_t id,
143 whio_epfs_inode * dest ); | 143 whio_epfs_inode * dest );
144 144
145 /** @internal 145 /** @internal
146 146
147 Returns the client data container object for fs. The client 147 Returns the client data container object for fs. The client
148 may alter its contents (if he is certain what he is doing) but 148 may alter its contents (if he is certain what he is doing) but
11 hidden lines
160 returned again on the next call to this function. 160 returned again on the next call to this function.
161 161
162 Error conditions include: 162 Error conditions include:
163 163
164 - !fs or !dest (whio_rc.ArgError). 164 - !fs or !dest (whio_rc.ArgError).
> 165
165 - No more free inodes (whio_rc.DeviceFullError). 166 - No more free inodes (whio_rc.DeviceFullError).
166 - Any potential errors via reading or writing the inode. | 167
167 - There's a whio_rc.RangeError for one theoretically impossible case.\ | 168 - Any potential errors via reading or writing the inode and
168 - markAsUsed is true but fs is opened for read-only more: whio_rc.AccessError. | 169 updating the free-inode list.
169 | 170
170 | 171 - markAsUsed is true but fs is opened for read-only more:
171 As of 20100310, this is a essentially an O(1) operation if we | 172 whio_rc.AccessError.
172 discount the i/o it must potentially do to update the inode | 173
173 free-list. If markAsUsed is true it must read, at most, 2 | 174 As of 20100310, the actual search is an O(1) operation but
174 additional inodes, update their free-list links, and flush | 175 there is (if markAsUsed is true) a small i/o overhead to
175 them. The ids (and therefore on-storage positions) of all | 176 update the inode free-list. If markAsUsed is true it must
176 inodes involved are known in advance, so this routine has to | 177 read, at most, 2 additional inodes, update their free-list
177 do no search-related i/o. Note that such re-linking pays no | 178 links, and flush them. The ids (and therefore on-storage
178 attention whatsoever to whether an inode or its neighbors are | 179 positions) of all inodes involved are known in advance, so
179 opened (and therefore have their inode state cached somewhere | 180 this routine has to do no search-related i/o. Note that such
180 in memory). If the API is used properly then by the time an | 181 re-linking pays no attention whatsoever to whether an inode or
181 inode can be opened, all links to/from the inode and the | 182 its neighbors are opened (and therefore have their inode state
182 free-list are severed. | 183 cached somewhere in memory!). If the API is used properly then
| 184 by the time an inode can be opened, all links to/from the
| 185 inode and the free-list are severed.
| 186
| 187 The down-side of the O(1) search guaranty is that if an i/o
| 188 error happens while updating the free-list links, it could
| 189 leave the free-list in an undefined state. That will either
| 190 lead to prematurely "running out" of inodes (when there are
| 191 actually some left) or to this routine taking longer to find
| 192 the next free inode (it double-checks to ensure that the
| 193 search led it to an unused inode). Or both. Probably both.
| 194
| 195 TODO: add the above-mentioned potential corruption case to the
| 196 future fsck-like functionality. We could theoretically rebuild
| 197 the free list from scratch by linearly scanning for
| 198 explicitly-marked used inodes.
183 */ 199 */
184 int whio_epfs_inode_next_free( whio_epfs * fs, whio_epfs_inode * dest, bool markAsUsed ); 200 int whio_epfs_inode_next_free( whio_epfs * fs, whio_epfs_inode * dest, bool markAsUsed );
185 201
186 /** @internal 202 /** @internal
187 203
865 hidden lines
1053 #ifdef __cplusplus 1069 #ifdef __cplusplus
1054 } /* extern "C" */ 1070 } /* extern "C" */
1055 #endif 1071 #endif
1056 1072
1057 #endif /* WANDERINGHORSE_NET_WHIO_EPFS_INTERNAL_H_INCLUDED */ 1073 #endif /* WANDERINGHORSE_NET_WHIO_EPFS_INTERNAL_H_INCLUDED */
Fossil version [2827d449a9] 2012-01-12 00:44:19.
This repository is part of the fossil.wanderinghorse.net collection.