Pen computing

From HandWiki
Samsung Galaxy Note phablets include a stylus, called "S-Pen".
Short description: Uses a stylus and tablet/touchscreen

Pen computing refers to any computer user-interface using a pen or stylus and tablet, over input devices such as a keyboard or a mouse.

Pen computing is also used to refer to the usage of mobile devices such as tablet computers, PDAs and GPS receivers. The term has been used to refer to the usage of any product allowing for mobile communication. An indication of such a device is a stylus or digital pen, generally used to press upon a graphics tablet or touchscreen, as opposed to using a more traditional interface such as a keyboard, keypad, mouse or touchpad.

Historically, pen computing (defined as a computer system employing a user-interface using a pointing device plus handwriting recognition as the primary means for interactive user input) predates the use of a mouse and graphical display by at least two decades, starting with the Stylator[1] and RAND Tablet[2] systems of the 1950s and early 1960s.

General techniques

User interfaces for pen computing can be implemented in several ways. Actual systems generally employ a combination of these techniques.

Pointing/locator input

The tablet and stylus are used as pointing devices, such as to replace a mouse. While a mouse is a relative pointing device (one uses the mouse to "push the cursor around" on a screen), a tablet is an absolute pointing device (one places the stylus where the cursor is to appear).

There are a number of human factors to be considered when actually substituting a stylus and tablet for a mouse. For example, it is much harder to target or tap the same exact position twice with a stylus, so "double-tap" operations with a stylus are harder to perform if the system is expecting "double-click" input from a mouse.

A finger can be used as the stylus on a touch-sensitive tablet surface, such as with a touchscreen.

Handwriting recognition

The tablet and stylus can be used to replace a keyboard, or both a mouse and a keyboard, by using the tablet and stylus in two modes:

  • Pointing mode: The stylus is used as a pointing device as above.
  • On-line Handwriting recognition mode: The strokes made with the stylus are analyzed as an "electronic ink" by software which recognizes the shapes of the strokes or marks as handwritten characters. The characters are then input as text, as if from a keyboard.

Different systems switch between the modes (pointing vs. handwriting recognition) by different means, e.g.

  • by writing in separate areas of the tablet for pointing mode and for handwriting-recognition mode.
  • by pressing a special button on the side of the stylus to change modes.
  • by context, such as treating any marks not recognized as text as pointing input.
  • by recognizing a special gesture mark.

The term "on-line handwriting recognition" is used to distinguish recognition of handwriting using a real-time digitizing tablet for input, as contrasted to "off-line handwriting recognition", which is optical character recognition of static handwritten symbols from paper.

Direct manipulation

The stylus is used to touch, press, and drag on simulated objects directly. The Wang Freestyle system[3] is one example. Freestyle worked entirely by direct manipulation, with the addition of electronic "ink" for adding handwritten notes.

Gesture recognition

This is the technique of recognizing certain special shapes not as handwriting input, but as an indicator of a special command.

For example, a "pig-tail" shape (used often as a proofreader's mark) would indicate a "delete" operation. Depending on the implementation, what is deleted might be the object or text where the mark was made, or the stylus can be used as a pointing device to select what it is that should be deleted. With Apple's Newton OS, text could be deleted by scratching in a zig-zag pattern over it.

Recent systems have used digitizers which can recognize more than one "stylus" (usually a finger) at a time, and make use of Multi-touch gestures.

The PenPoint OS was a special operating system which incorporated gesture recognition and handwriting input at all levels of the operating system. Prior systems which employed gesture recognition only did so within special applications, such as CAD/CAM applications[4][5] or text processing.[6]

History

Pen computing has very deep historical roots. For example, the first patent for an electronic device used for handwriting, the telautograph, was granted in 1888.[7] What is probably the first patent for a system that recognized handwritten characters by analyzing the handwriting motion was granted in 1915.[8] Around 1954 Douglas T Ross, working on the Whirlwind computer at MIT, wrote the "first hand-drawn graphics input program to a computer".[9] The first publicly demonstrated system using a tablet and handwriting text recognition instead of a keyboard for working with a modern digital computer dates to 1956.[10]

In addition to many academic and research systems, there were several companies with commercial products in the 1980s: Pencept, Communications Intelligence Corporation, and Linus were among the best known of a crowded field. Later, GO Corp. brought out the PenPoint OS operating system for a tablet PC product: one of the patents from GO corporation was the subject of recent infringement lawsuit concerning the Tablet PC operating system.[11]

The following timeline list gives some of the highlights of this history:

  • Before 1950
    • 1888: U.S. Patent granted to Elisha Gray on electrical stylus device for capturing handwriting.[7][12]
    • 1915: U.S. Patent on handwriting recognition user interface with a stylus.[8][13]
    • 1942: U.S. Patent on touchscreen for handwriting input.[14][15]
    • 1945: Vannevar Bush proposes the Memex, a data archiving device including handwriting input, in an essay As We May Think.[16]
  • 1950s
    • Tom Dimond demonstrates the Styalator electronic tablet with pen for computer input and handwriting recognition.[10]
  • Early 1960s
  • Late 1960s
    • Alan Kay of Xerox PARC proposed a notebook using pen input called Dynabook: however, the device is never constructed.
  • 1971
    • Touchscreen interface developed at SLAC.[19]
  • 1979
    • Fairlight CMI, an early commercial digital sampling workstation
  • 1982
    • Pencept of Waltham, Massachusetts markets a general-purpose computer terminal using a tablet and handwriting recognition instead of a keyboard and mouse.[20]
    • Cadre System markets the Inforite point-of-sale terminal using handwriting recognition and a small electronic tablet and pen.[21]
  • 1985:
    • Pencept[22][23] and CIC[24] both offer PC computers for the consumer market using a tablet and handwriting recognition instead of a keyboard and mouse. Operating system is MS-DOS.
  • 1989
    • The first commercially available tablet-type portable computer was the GRiDPad 1900[25] from GRiD Systems, released in September. It ran GRiDPen (later released as PenRight), a graphic system with pen input and handwriting recognition running on MS-DOS.
    • Wang Laboratories introduces Freestyle. Freestyle was an application that would do a screen capture from an MS-DOS application, and let the user add voice and handwriting annotations. It was a sophisticated predecessor to later note-taking applications for systems like the Tablet PC. The operating system was MS-DOS.[26]
  • 1991
    • The Momenta Pentop was released.[27][28]
    • GO Corp announced a dedicated operating system, called PenPoint OS, featuring control of the operating system desktop via handwritten gesture shapes.[29][30] Gestures included "flick" gestures in different directions, check-marks, cross-outs, pig-tails, and circular shapes, among others.
    • Portia Isaacsen of Future Computing estimates the total annual market for pen computers such as those running the PenPoint OS to be on the order of $500 Million.
    • NCR released model 3125 pen computer running MS-DOS, Penpoint or Pen Windows.[31]
    • The Apple Newton entered development; although it ultimately became a PDA, its original concept (which called for a larger screen and greater sketching capabilities) resembled that of a tablet PC.
    • Sam Tramiel of Atari Corp. presented the "ST-Pad" (codenamed "STylus") at the CeBIT '91 in Hanover, Germany . The computer never went into production.
  • 1992
    • GO Corp shipped PenPoint and IBM announced IBM 2125 pen computer (the first IBM model named "ThinkPad") in April.[32]
    • Microsoft releases Windows for Pen Computing as a response to the PenPoint OS.
  • 1993
    • IBM releases the ThinkPad, IBM's first commercialized portable tablet computer product available to the consumer market, as the IBM ThinkPad 750P and 360P[33]
    • Apple Computer announces the Newton PDA, also known as the Apple MessagePad, which includes handwriting recognition with a stylus.
    • Amstrad release the "PenPad" or PDA600, a similar pen-based device. It did not achieve commercial success.[34]
    • AT&T introduced the EO Personal Communicator combining PenPoint with wireless communications.
    • BellSouth released the IBM Simon Personal Communicator, an analog cellphone using a touch-screen and display. It did not include handwriting recognition, but did permit users to write messages and send them as faxes on the analog cellphone network, and included PDA and Email features.
  • 1999
    • The "QBE" pen computer created by Aqcess Technologies wins Comdex Best of Show.[35]
  • 2000
    • The "QBE Vivo" pen computer created by Aqcess Technologies ties for Comdex Best of Show.
  • 2001
    • Bill Gates of Microsoft demonstrates first public prototype of a Tablet PC (defined by Microsoft as a pen-enabled computer conforming to hardware specifications devised by Microsoft and running a licensed copy of Windows XP Tablet PC Edition)[36] at Comdex.
    • Wacom introduces the Cintiq pen-based tablet platform for professional artists.[37]
  • 2003
  • 2005
  • 2006
    • Windows Vista released for general availability. Vista included the functionality of the special Tablet PC edition of Windows XP.
  • 2008
    • In April 2008, as part of a larger federal court case, the gesture features of the Windows/Tablet PC operating system and hardware were found to infringe on a patent by GO Corp. concerning user interfaces for pen computer operating systems.[11] Microsoft's acquisition of the technology is the subject of a separate lawsuit.[41][42]
    • HP releases the second MultiTouch capable tablet: the HP TouchSmart tx2z.[43]
  • 2011
  • 2012
    • Microsoft releases the Microsoft Surface Pro hybrid tablet/laptop with an optional Surface Pen.
  • 2013
  • 2015
    • Apple releases the Apple Pencil for the iPad Pro which includes pressure sensitivity and angle detection.
  • 2021
    • Samsung releases the Samsung Galaxy S21 Ultra, the first Samsung Galaxy S phone to bring the S Pen support found in the Samsung Galaxy Note Series, alongside the release of the S Pen Pro as the separate high-end stylus to have a compatible with the smartphones, tablets and PCs that supports the S Pen.
    • In August 2021, as the discontinuation of the entire Samsung Galaxy Note Series, Samsung releases the Samsung Galaxy Z Fold 3, the first Galaxy Z Fold foldable phone to bring the S Pen support, alongside the release of the S Pen Fold Edition as the separate stylus to have exclusively compatible with the Galaxy Z Fold 3 onwards to avoid taking the screen damage rather than using the other stylus pens and the S Pen Pro without the Z Fold compatibility.

See also

References

  1. ↑ Dimond, T.L. (1957-12-01), Devices for reading handwritten characters, Proceedings of Eastern Joint Computer Conference, pp. 232–237, http://ruetersward.com/pens/biblio70.html#Dimond57 
  2. ↑ Groner, G.F. (August 1966), Real-Time Recognition of Handprinted Text, Memorandum RM-5016-ARPA, RAND Corporation, http://users.erols.com/rwservices/pens/biblio70.html#Groner66a 
  3. ↑ WANG Freestyle demo, Wang Laboratories, 1989, http://ruetersward.com/pens/images.html#WangFreestyle, retrieved 2008-09-22 
  4. ↑ Computerized Graphic Processing System: System User's Manual, Applicon Incorporated, 1973-09-01, http://users.erols.com/rwservices/pens/biblio75.html#Applicon73 
  5. ↑ Newman, W.M. (1973-09-01), The Ledeen Character Recognizer, Principles of Interactive Computer Graphics, McGraw-Hill, pp. 575–582, http://ruetersward.com/pens/biblio75.html#NewmanWM73a 
  6. ↑ Coleman, Michael L. (1969), Text editing on a graphic display device using hand-drawn proofreader's symbols, from Pertinent Concepts in Computer Graphics: Proceedings of the 2nd University of Illinois Conference on Computer Graphics, University of Illinois Press, http://users.erols.com/rwservices/pens/biblio70.html#Coleman69 
  7. ↑ 7.0 7.1 Gray, Elisha (1888-07-31), Telautograph, United States Patent 386,815 (full image), http://www.freepatentsonline.com/386815.pdf 
  8. ↑ 8.0 8.1 Goldberg, H.E. (1915-12-28), Controller, United States Patent 1,117,184 (full image), http://www.freepatentsonline.com/1117184.pdf 
  9. ↑ Ross, Doug (1989), Retrospectives 1: The early years in computer graphics, SIGGRAPH 89 Proceedings, pp. 27–28, http://excelsior.biosci.ohio-state.edu/~carlson/history/PDFs/p19-hurst.pdf 
  10. ↑ 10.0 10.1 Dimond, Tom (1957-12-01), Devices for reading handwritten characters, Proceedings of Eastern Joint Computer Conference, pp. 232–237, http://users.erols.com/rwservices/pens/biblio70.html#DimondTL57a, retrieved 2008-08-23 
  11. ↑ 11.0 11.1 Mintz, Jessica (2008-04-04), Microsoft to Appeal $367M Patent Ruling, The Associated Press, https://www.usatoday.com/tech/products/2008-04-04-2507619152_x.htm, retrieved 2008-09-04 
  12. ↑ Gray (1888-07-31), Telautograph, United States Patent 386,815, http://ruetersward.com/pens/biblio70.html#Gray1888b 
  13. ↑ Goldberg, H.E. (1915-12-28), Controller, United States Patent 1,117,184, http://users.erols.com/rwservices/pens/biblio70.html#GoldbergHE15 
  14. ↑ Moodey, H.C. (1942-12-27), Telautograph System, United States Patent 2,269,599, http://users.erols.com/rwservices/pens/biblio70.html#Moodey40 
  15. ↑ Moodey, H.C. (1942-12-27), Telautograph System, United States Patent 2,269,599 (full image), http://www.freepatentsonline.com/2269599.pdf 
  16. ↑ Bush, Vannevar (1945-07-15), As We May Think, The Atlantic Monthly, http://ruetersward.com/pens/biblio70.html#BushV45 
  17. ↑ RAND Tablet, 1961-09-01, http://users.erols.com/rwservices/pens/biblio70.html#RAND61 
  18. ↑ 50 Years of Looking Forward, RAND Corporation, 1998-09-01, http://www.rand.org/publications/randreview/issues/rr.fall.98/50.html 
  19. ↑ Fryberger, D. (March 1972), An Innovation in Control Panels for Large Computer Control Systems, Proc. 4th IEEE Particle Accelerator Conference, Chicago IL USA, March 1..3, 1971, pp. 414–417, http://ruetersward.com/pens/biblio75.html#FrybergerD71a 
  20. ↑ Pencept Penpad (TM) 200 Product Literature, Pencept, Inc., 1982-08-15, http://users.erols.com/rwservices/pens/biblio83.html#Pencept83 
  21. ↑ Inforite Hand Character Recognition Terminal, Cadre Systems Limited, England, 1982-08-15, http://ruetersward.com/pens/biblio83.html#Inforite82 
  22. ↑ Users Manual for Penpad 320, Pencept, Inc., 1984-06-15, http://users.erols.com/rwservices/pens/biblio85.html#Pencept84d 
  23. ↑ Software Control at the Stroke of a Pen, Pencept, Inc., 1985, http://www.open-video.org/details.php?videoid=8023&surrogate=storyboard, retrieved 2009-05-21 
  24. ↑ Handwriter (R) GrafText (TM) System Model GT-5000, Communication Intelligence Corporation, 1985-01-15, http://users.erols.com/rwservices/pens/biblio85.html#CIC85 
  25. ↑ The BYTE Awards: GRiD System's GRiDPad, BYTE Magazine, Vol 15. No 1, 1990-01-12, p. 285, http://ruetersward.com/pens/biblio90.html#GridPad90a 
  26. ↑ Wang Laboratories (1989), The Wang Freestyle System, WANG Laboratories, http://ruetersward.com/pens/biblio90.html#WangLaboratories89a 
  27. ↑ Lempesis, Bill (May 1990), What's New in Laptops and Pen Computing, Flat Panel Display News, http://users.erols.com/rwservices/pens/biblio90.html#Momenta90 
  28. ↑ "Momenta Corporation 1/40 Pentop Computer - RICM". https://sites.google.com/a/ricomputermuseum.org/home/Home/small-systems-at-ricm/momenta-corporation-140-pen-computer. 
  29. ↑ Agulnick, Todd (1994-09-13), Control of a computer through a position-sensed stylus, United States Patent 5,347,295, http://users.erols.com/rwservices/pens/biblio95.html#Agulnick94 
  30. ↑ Agulnick, Todd (1994-09-13), Control of a computer through a position-sensed stylus, United States Patent 5,347,295 (full image), http://www.freepatentsonline.com/5347295.pdf 
  31. ↑ NCR announces pen-based computer press release, http://findarticles.com/p/articles/mi_m0NEW/is_1991_June_24/ai_10957018, retrieved 2007-04-20 
  32. ↑ Penpoint OS shipping press release, http://findarticles.com/p/articles/mi_m0NEW/is_1992_April_17/ai_12165379, retrieved 2007-04-20 
  33. ↑ "The history of ThinkPad". Archived from the original on 2007-04-29. https://web.archive.org/web/20070429123812/http://www.pc.ibm.com/us/thinkpad/anniversary/history.html. 
  34. ↑ Amstrad PenPad PDA600. Computing History (2011-10-21). Retrieved on 2013-12-09.
  35. ↑ Trends at COMDEX Event 1999, http://www.guiart.fi/gobr01en.htm, retrieved 2008-08-11 
  36. ↑ Microsoft (2005), Windows XP Tablet PC Edition 2005 Hardware Requirements, www.microsoft.com, http://ruetersward.com/pens/biblio10.html#Microsoft06i, retrieved 2009-03-14 
  37. ↑ "Annual Report 2009". Wacom. http://investors.wacom.com/media/files/investor-relations/2009-english/20090626annualreport2009.pdf?vs=1&d=00010101T000000. 
  38. ↑ Fingerworks, Inc. (2003), iGesture Game Mode Guide, www.fingerworks.com, http://users.erols.com/rwservices/pens/biblio05.html#Fingerworks03, retrieved 2009-04-30 
  39. ↑ MacEssentials (2007-08-02), Rubrik Apple: Das Lexikon der Fingersprache, www.mac-essentials.de, http://users.erols.com/rwservices/pens/biblio10.html#MacEssentials07a, retrieved 2009-05-16 
  40. ↑ Druin, Allison (2009-03-16) (in en). Mobile Technology for Children: Designing for Interaction and Learning. Morgan Kaufmann. pp. 177. ISBN 978-0-08-095409-7. https://books.google.com/books?id=Pv18x3wIVzIC&pg=PA177. 
  41. ↑ Fried, Ina (July 6, 2005). "Go files antitrust suit against Microsoft". CNET. https://www.cnet.com/tech/tech-industry/go-files-antitrust-suit-against-microsoft/. 
  42. ↑ A New Antitrust Lawsuit - Go Corp. v. Microsoft. Groklaw (2005-07-04). Retrieved on 2013-12-09.
  43. ↑ HP TouchSmart tx2z, HP, http://www.shopping.hp.com/webapp/shopping/computer_can_series.do?storeName=computer_store&category=notebooks&a1=Category&v1=Mobility&series_name=tx2z_series, retrieved 2008-11-28 

External links



Retrieved from "https://handwiki.org/wiki/index.php?title=Pen_computing&oldid=3364877"