Bài giảng Unit OS6: Device Management - Windows I/O Processing

Nội dung text: Bài giảng Unit OS6: Device Management - Windows I/O Processing

1. Unit OS6: Device Management 6.3. Windows I/O Processing Windows Operating System Internals - by David A. Solomon and Mark E. Russinovich with Andreas Polze
2. Copyright Notice © 2000-2005 David A. Solomon and Mark Russinovich These materials are part of the Windows Operating System Internals Curriculum Development Kit, developed by David A. Solomon and Mark E. Russinovich with Andreas Polze Microsoft has licensed these materials from David Solomon Expert Seminars, Inc. for distribution to academic organizations solely for use in academic environments (and not for commercial use) 2
3. Roadmap for Section 6.3 DriverDriver andand DeviceDevice ObjectsObjects I/OI/O RequestRequest PacketsPackets (IRP)(IRP) ProcessingProcessing DriverDriver LayeringLayering andand FilteringFiltering Plug-and-PlayPlug-and-Play (PnP)(PnP) andand PowerPower ManagerManager OperationOperation MonitoringMonitoring I/OI/O ActivityActivity withwith FilemonFilemon 3
4. Driver Object AA driverdriver objectobject representsrepresents aa loadedloaded driverdriver Names are visible in the Object Manager namespace under \Drivers A driver fills in its driver object with pointers to its I/O functions e.g. open, read, write When you get the “One or More Drivers Failed to Start” message its because the Service Control Manager didn’t find one or more driver objects in the \Drivers directory for drivers that should have started 4
5. Device Objects AA devicedevice objectobject representsrepresents anan instanceinstance ofof aa devicedevice Device objects are linked in a list off the driver object A driver creates device objects to represent the interface to the logical device, so each generally has a unique name visible under \Devices Device objects point back at the Driver object 5
6. Driver and Device Objects Driver Object \Device\TCP \Device\UDP \Device\IP \TCPIP Open Open( ) Write Read Read( ) Write( ) Dispatch Table Loaded Driver Image TCP/IP Drivers Driver and Device Objects 6
7. File Objects Represents open instance of a device (files on a volume are virtual devices) Applications and drivers “open” devices by name The name is parsed by the Object Manager When an open succeeds the object manager creates a file object to represent the open instance of the device and a file handle in the process handle table A file object links to the device object of the “device” which is opened File objects store additional information File offset for sequential access File open characteristics (e.g. delete-on-close) File name Accesses granted for convenience 7
8. I/O Request Packets System services and drivers allocate I/O request packets to describe I/O A request packet contains: File object at which I/O is directed I/O characteristics (e.g. synchronous, non-buffered) Byte offset Length Buffer location The I/O Manager locates the driver to which to hand the IRP by following the links: File Object Device Object Driver Object 8
9. Putting it Together: Request Flow Process DeviceIoControl User Mode Kernel Mode Dispatch Table NtDeviceIoControlFile File Device Driver Object Object Object Handle Table IRP DispatchDeviceControl( DeviceObject, Irp ) Driver Code 9
10. I/O Request Packet 1)An application writes Environment a file to the printer, subsystem or passing a handle to DLL User mode the file object Kernel mode Services 2)The I/O manager I/O manager creates an IRP and IRP stack initializes first stack location location IRP header WRITE File Device Driver parameters object object object 3)The I/O manager uses the driver object to locate the WRITE dispatch Dispatch DPC Start I/O ISR routine and calls it, routine(s) routine passing the IRP Device Driver 10
11. IRP data IRP consists of two parts: Fixed portion (header): Type and size of the request Whether request is synchronous or asynchronous Pointer to buffer for buffered I/O State information (changes with progress of the request) One or more stack locations: Function code Function-specific parameters Pointer to caller‘s file object While active, IRPs are stored in a thread-specific queue I/O system may free any outstanding IRPs if thread terminates 11
12. I/O Processing – synch. I/O to a single-layered driver 1. The I/O request passes through a subsystem DLL 2. The subsystem DLL calls the I/O manager‘s NtWriteFile() service 3. I/O manager sends the request in form of an IRP to the driver (a device driver) 4. The driver starts the I/O operation 5. When the device completes the operation and interrupts the CPU, the device driver services the int. 6. The I/O manager completes the I/O request 12
13. Completing an I/O request Servicing an interrupt: ISR schedules Deferred Procedure Call (DPC); dismisses int. DPC routine starts next I/O request and completes interrupt servicing May call completion routine of higher-level driver I/O completion: Record the outcome of the operation in an I/O status block Return data to the calling thread – by queuing a kernel-mode Asynchronous Procedure Call (APC) APC executes in context of calling thread; copies data; frees IRP; sets calling thread to signaled state I/O is now considered complete; waiting threads are released 13
14. Flow of Interrupts 0 2 3 Peripheral Device CPU Interrupt Controller Controller n CPU Interrupt Service Table ISR Address Read from device Raise IRQL Spin Lock Grab Spinlock Acknowledge- Interrupt Dispatch Code Drop Spinlock Request DPC Lower IRQL Interrupt KiInterruptDispatch Driver ISR Object 14
15. Servicing an Interrupt: Deferred Procedure Calls (DPCs) Used to defer processing from higher (device) interrupt level to a lower (dispatch) level Also used for quantum end and timer expiration Driver (usually ISR) queues request One queue per CPU. DPCs are normally queued to the current processor, but can be targeted to other CPUs Executes specified procedure at dispatch IRQL (or “dispatch level”, also “DPC level”)level”) whenwhen allall higher-IRQLhigher-IRQL workwork (interrupts)(interrupts) completedcompleted Maximum times recommended: ISR: 10 usec, DPC: 25 usec See queue head DPC object DPC object DPC object 15
16. Delivering a DPC 1. Timer expires, kernel DPC routines can‘t DPC queues DPC that will Interrupt release all waiting threads dispatch table assume what Kernel requests SW int. process address high space is currently Power failure mapped 2. DPC interrupt occurs 3. After DPC interrupt, when IRQL drops below control transfers to dispatch/DPC level thread dispatcher DPCDPC DPC Dispatch/DPC dispatcher DPC queue APC Low DPC routines can call kernel functions 4. Dispatcher executes each DPC but can‘t call system services, generate routine in DPC queue page faults, or create or wait on objects 16
17. I/O Completion: Asynchronous Procedure Calls (APCs) Execute code in context of a particular user thread APC routines can acquire resources (objects), incur page faults, call system services APC queue is thread-specific User mode & kernel mode APCs Permission required for user mode APCs Executive uses APCs to complete work in thread space Wait for asynchronous I/O operation Emulate delivery of POSIX signals Make threads suspend/terminate itself (env. subsystems) APCs are delivered when thread is in alertable wait state WaitForMultipleObjectsEx(), SleepEx() 17
18. Asynchronous Procedure Calls (APCs) Special kernel APCs Run in kernel mode, at IRQL 1 Always deliverable unless thread is already at IRQL 1 or above Used for I/O completion reporting from “arbitrary thread context” Kernel-mode interface is linkable, but not documented “Ordinary” kernel APCs Always deliverable if at IRQL 0, unless explicitly disabled (disable with KeEnterCriticalRegion) User mode APCs Used for I/O completion callback routines (see ReadFileEx, WriteFileEx); also, QueueUserApc Only deliverable when thread is in “alertable wait” K Thread APC objects Object U 18
19. Driver Layering and Filtering To divide functionality across Process drivers, provide added value, etc. User Mode Only the lowest layer talks to Kernel Mode the I/O hardware “Filter drivers” attach their devices System Services to other devices File System Driver They see all requests first and can manipulate them I/O Manager Volume Example filter drivers: Manager File system filter driver Driver IRP Bus filter driver Disk Driver 19
20. Driver Filtering: Volume Shadow Copy New to XP/Server 2003 Addresses the “backup open files” problem Volumes can be “snapshotted” Allows “hot backup” (including open files) Applications can tie in with mechanism to ensure consistent snapshots Database servers flush transactions Windows components Volsnap is the built-in provider: such as the Registry •Built into Windows XP/Server 2003 flush data files •Implements copy-on-write snapshots Different snapshot providers •Saves volume changes in files on the can implement different snapshot volume mechanisms: •Uses defrag API to determine where the file Copy-on-write is and where paging file is to avoid tracking Copy-on-write their changes Mirroring 20
21. Volume Snapshots Writers Backup Application 5. Backup application 2. Writers told saves data from volume Oracle to freeze Shadow copies activity 1. Backup application requests Volume Shadow shadow copy Copy Service SQL 4. Writers told to resume (“thaw”) activity 3. Providers asked to create volume shadow copies Volume Shadow Copy Driver Mirror provider (volsnap.sys) Providers 21
22. Volsnap.sys Backup Application Application Shadow Volume C: C: File System Driver Volsnap.sys Backup read of sector c Application read of sector c a b c All reads of sector d Snapshot a d b c 22
23. Shadow Copies of Shared Folders WhenWhen enabled,enabled, ServerServer 20032003 usesuses shadowshadow copycopy toto periodicallyperiodically createcreate snapshotssnapshots ofof volumesvolumes ScheduleSchedule andand spacespace usedused isis configurableconfigurable 23
24. Shadow Copies on Shared Folders Shadow copies are only exposed as network shares Clients may install an Explorer extension that integrates with the file server and let’s them View the state of folders and files within a snapshot Rollback individual folders and files to a snapshot 24
25. The PnP Manager InIn NTNT 4.04.0 eacheach devicedevice driverdriver isis responsibleresponsible forfor enumeratingenumerating allall supportedsupported bussesbusses inin searchsearch ofof devicesdevices theythey supportsupport AsAs ofof WindowsWindows 2000,2000, thethe PnPPnP ManagerManager hashas busbus driversdrivers enumerateenumerate theirtheir bussesbusses andand informinform itit ofof presentpresent devicesdevices IfIf thethe devicedevice driverdriver forfor aa devicedevice notnot alreadyalready presentpresent onon thethe system,system, thethe PnPPnP ManagerManager inin thethe kernelkernel informsinforms thethe user-modeuser-mode PnPPnP ManagerManager toto startstart thethe HardwareHardware WizardWizard 25
26. The PnP Manager Once a device driver is located, the PnP Manager determines if the driver is signed If the driver isis notnot signed,signed, thethe system’ssystem’s driverdriver signing policy determines whether or not the driver is installed After loading a driver, the PnP Manager calls the driver’s AddDevice entry point The driver informs the PnP Manager of the device’s resource requirements The PnP Manager reconfigures other devices to accommodate the new device 26
27. The PnP Manager Enumeration is recursive, and directed by bus drivers Bus drivers identify device on a bus As busses and devices are registered, a device tree is constructed, and filled in with devices Key- board Video Disk USB PCI Battery ACPI Device Tree Root 27
28. Resource Arbitration Devices require system hardware resources to function (e.g. IRQs, I/O ports) The PnP Manager keeps track of hardware resource assignments If a device requires a resource that’s already been assigned, the PnP Manager tries to reassign resources in order to accommodate Example: 1. Device 1 can use IRQ 5 or IRQ 6 2. PnP Manager assigns it IRQ 5 3. Device 2 can only use IRQ 5 4. PnP Manager reassigns Device 1 IRQ 6 5. PnP Manager assigns Device 2 IRQ 5 28
29. Plug and Play (PnP) State Transitions PnP manager recognizes hardware, allocates resources, loads driver, notifies about config. changes Query-remove Not started command Remove Start-device Pending command command remove Started Removed Start-device Query-stop Remove command command Surprise command remove Pending stop Surprise-remove Stop command command Stopped Device Plug and Play state transitions 29
30. The Power Manager A system must have an ACPI-compliant BIOS for full compatibility (APM gives limited power support) A number of factors guide the Power Manager’s decision to change power state: System activity level System battery level Shutdown, hibernate, or sleep requests from applications User actions, such as pressing the power button Control Panel power settings The system can go into low power modes, but it requires the cooperation of every device driver - applications can provide their input as well 30
31. The Power Manager There are different system power states: On Everything is fully on Standby Intermediate states Lower standby states must consume less power than higher ones Hibernating Save memory to disk in a file called hiberfil.sys in the root directory of the system volume Off All devices are off Device drivers manage their own power level Only a driver knows the capabilities of their device Some devices only have “on” and “off”, others have intermediate states Drivers can control their own power independently of system power Display can dim, disk spin down, etc. 31
32. Power Manager based on the Advanced Configuration and Power Interface (ACPI) State Power Consumption Software Resumption HW Latency S0 (fully on) Maximum Not applicable None S1 (sleeping) Less than S0, System resumes where it left Less than 2 more than S2 off (returns to S0) sec. S2 (sleeping) Less than S1, System resumes where it left 2 or more more than S3 off (returns to S0) sec. S3 (sleeping) Less than S2, System resumes where it left Same as S2 processor is off off (returns to S0) S4 (sleeping) Trickle current to power System restarts from Long and button and wake hibernate file and resumes undefined circuitry where it left off (returns to S0) S5 (fully off) Trickle current to System boot Long and power button undefined System Power-State Definitions 32
33. Troubleshooting I/O Activity Filemon can be a great help to understand and troubleshooting I/O problems Two basic techniques: Go to end of log and look backwards to where problem occurred or is evident and focused on the last things done Compare a good log with a bad log Often comparing the I/O activity of a failing process with one that works may point to the problem Have to first massage log file to remove data that differs run to run Delete first 3 columns (they are always different: line #, time, process id)id) Easy to do with Excel by deleting columns Then compare with FC (built in tool) or Windiff (Resource Kit) 33
34. Filemon ## operationoperation numbernumber Process:Process: imageimage namename ++ processprocess idid Request:Request: internalinternal I/OI/O requestrequest codecode Result:Result: returnreturn codecode fromfrom I/OI/O operationoperation Other:Other: flagsflags passedpassed onon I/OI/O requestrequest 34
35. Using Filemon Start/stopStart/stop logginglogging (Control/E)(Control/E) ClearClear displaydisplay (Control/X)(Control/X) OpenOpen ExplorerExplorer windowwindow toto folderfolder containingcontaining file:file: Double click on a line does this FindFind –– findsfinds texttext withinwithin windowwindow SaveSave toto loglog filefile AdvancedAdvanced modemode NetworkNetwork optionoption 35
36. What Filemon Monitors ByBy defaultdefault FilemonFilemon tracestraces allall filefile I/OI/O to:to: Local non-removable media Network shares StoresStores allall outputoutput inin listviewlistview Can exhaust virtual memory in long runs You can limit captured data with history depth YouYou cancan limitlimit whatwhat isis monitored:monitored: What volumes to watch in Volumes menu What paths and processes to watch in Filter dialog What operations to watch in Filter dialog (reads, writes, successes and errors) 36
37. Filemon Filtering and Highlighting Include and exclude filters are substring matches against the process and path columns Exclude overrides include filter Be careful that you don’t exclude potentially useful data Capture everything and save the log Then apply filters (you can always reload the log) Highlight matches all columns 37
38. Basic vs Advanced Mode BasicBasic modemode massagesmassages outputoutput toto bebe sysadminsysadmin friendlyfriendly andand targettarget commoncommon troubleshootingtroubleshooting ThingsThings youyou don’tdon’t seesee inin BasicBasic mode:mode: Raw I/O request names Various internal file system operations Activity in the System process Page file I/O Filemon file system activity 38
39. Understanding Disk Activity Use Filemon to see why you’re hard disk is crunching Process performance counters show I/O activity, but not to where System performance counters show which disks are being hit, but not which files or which process Filemon pinpoints which file(s) are being accessed, by whom, and how frequently You can also use Filemon on a server to determine which file(s) were being accessed most frequently Import into Excel and make a pie chart by file name or operation type Move heavy-access files to a different disk on a different controller 39
40. Polling and File Change Notification Many applications respond to file and directory changes A poorly written application will “poll” for changes A well-written application will request notification by the system of changes Polling for changes causes performance degradation Context switches including TLB flush Cache invalidation Physical memory usage CPU usage Alternative: file change notification When you run Filemon on an idle system you should only see bursty system background activity Polling is visible as periodic accesses to the same files and directories File change notification is visible as directory queries that have no result 40
41. Example: Word Crash While typing in the document Word XP would intermittently close without any error message To troubleshoot ran Filemon on user’s system SetSet thethe historyhistory depthdepth toto 10,00010,000 AskedAsked useruser toto sendsend FilemonFilemon loglog whenwhen WordWord exitedexited 41
42. Solution: Word Crash WorkingWorking backwards,backwards, thethe firstfirst “strange”“strange” oror unexplainableunexplainable behaviorbehavior areare thethe constantconstant readsreads pastpast endend ofof filefile toto MSSP3ES.LEXMSSP3ES.LEX UserUser lookedlooked upup whatwhat .LEX.LEX filefile waswas Related to Word proofing tools Uninstalled and reinstalled proofing tools & problem went away 42
43. Example: Useless Excel Error Message Excel reports an error “Unable to read file" when starting 43
44. Solution: Useless Excel Error Message FilemonFilemon tracetrace showsshows ExcelExcel readingreading filefile inin XLStartXLStart folderfolder All Office apps autoload files in their start folders ShouldShould havehave reported:reported: Name and location of file Reason why it didn’t like it 44
45. Further Reading MarkMark E.E. RussinovichRussinovich andand DavidDavid A.A. Solomon,Solomon, MicrosoftMicrosoft WindowsWindows Internals,Internals, 4th4th Edition,Edition, MicrosoftMicrosoft Press,Press, 2004.2004. I/O Processing (from pp. 561) The Plug and Play (PnP) Manager (from pp. 590) The Power Manager (from pp. 607) Troubleshooting File System Problems (from pp. 711) 45
46. Source Code References WindowsWindows ResearchResearch KernelKernel sourcessources \base\ntos\io – I/O Manager \base\ntos\inc\io.h – additional structure/type definitions \base\ntos\verifer – Driver Verifier \base\ntos\inc\verifier.h – additional structure/type definitions 46