1 Overview and Introduction 1

1.1 Why Meme Media? 1

1.2 How Do Meme Media Change the Reuse of Web Contens? 3

1.3 How Do Meme Media Work? 7

1.4 Frequently Asked Questions and Limitations 8

1.5 Organization of this Book 9

2 Knowledge Media and Meme Media 11

2.1 Introduction to Knowledge Media and Meme Media 11

2.1.1 Information Life Cycle and Knowledge Media 13

2.1.2 Artificial Intelligence Versus Knowledge Media 15

2.1.3 Meme Media for All Users 17

2.1.4 Meme Media and Compound Documents 20

2.1.5 Objects and Media 21

2.1.6 Multimedia, Hypermedia, and Meme Media 26

2.1.7 Meme Media and Meme Pools 27

2.2 From Information Technologies to Media Technologies 27

2.2.1 Information Architectures and Media Architectures 28

2.2.2 Roles of Media 29

2.2.3 History of Books 30

2.2.4 From Information Processing to Social Information Infrastructure 31

2.2.5 Editing, Distribution, and Management 31

2.2.6 Superdistribution of Knowledge Media 00

2.3 Summary 34

References 34

3 Augmentation Media Architectures and Technologies— 35

A Brief Survey

3.1 History and Evolution of Augmentation Media 36

3.1.1 Pioneers 36

3.1.2 Evolution of Personal-Augmentation Media 38

3.1.3 The Evolution of Group-Augmentation Media 40

3.1.4 The Evolution of Organization-Augmentation Media 42

3.1.5 The Evolution of Social-Augmentation Media 44

3.2 History and Evolution of Knowledge-Media Architectures 45

3.2.1 Compound-Document Architectures 45

3.2.2 Media Toolkit Systems 47

3.2.3 IntelligentPad as a Meme Media System 48

3.3 Meme Media and their Applications 48

3.3.1 Office-Information Systems 49

3.3.2 Scientific Publication 50

3.3.3 Education Tools 51

3.3.4 Networked Multimedia Systems 52

3.3.5 Geographic Information Systems 52

3.3.6 Design Applications 53

3.3.7 DIY Software for Client Systems 53

3.4 Web Technologies and Meme Media 55

3.4.1 Open Hypermedia Systems 55

3.4.2 Client-Side Web Programs and XML 56

3.4.3 Server-Side Web Programs 58

3.5 Summary 59

References 60

4 An Outline of IntelligentPad and Its Development History 67

4.1 Brief Introduction to IntelligentPad 67

4.1.1 The Motivation for Our Project 68

4.1.2 Synthetic Media Architecture 69

4.1.3 Meme Media Architecture 70

4.2 IntelligentPad Architecture 70

4.2.1 Pad Architecture 71

4.2.2 Paste Operation and Slot Connection 71

4.2.3 IntelligentPad as a Meta-Tool 74

4.2.4 Pads as Meme Media 77

4.3 Worldwide Marketplace Architectures for Pads 80

4.4 End-User Computing and Media Toolkit System 81

4.5 Open Cross-Platform Reusability 85

4.6 Reediting and Redistribution by End-Users 86

4.7 Extension toward 3D Representation Media 88

4.8 Summary 89

References 90

5 Object Orientation and MVC 92

5.1 Object-Oriented System Architecture—A Technical Introduction 93

5.2 Class Refinement and Prototyping 94

5.3 Model, View, Controller 95

5.3.1 MVC Construct 95

5.3.2 Dependencies in MVC 96

5.3.3 Pluggable VC 98

5.4 Window Systems and Event Dispatching 99

5.4.1 Event Dispatching 100

5.4.2 Redrawing of Overlaid Windows 102

5.4.3 From Windowpanes to Visual Objects 103

5.5 Summary 103

References 104

6 Component Integration 106

6.1 Object Reusability 107

6.2 Components and Application Linkage 107

6.3 Compound Documents and Object Embedding/Linking 113

6.4 Generic Components 114

6.5 What to Reuse—Components or Sample Compositions? 115

6.6 Reuses and Maintenance 116

6.7 Integration of Legacy Software 118

6.8 Distributed Component Integration and Web Technologies 119

6.8.1 CORBA and Application Server Component Technologies 119

6.8.2 Web Services and their Integration 121

6.8.3 The Internet as a Platform and Universal Document Interface 122

6.8.4 The Internet as Shared Memory Spaces for Objects 123

6.8.5 Distributed Object Technologies and Meme Media Components 124

6.9 Summary 125

References 125

7 Meme Media Architecture 128

7.1 Current Megatrends in Computer Systems 128

7.2 Primitive Media Objects 129

7.2.1 Wrapper Architecture 130

7.2.2 Frame Architecture of Each Pad 131

7.2.3 MVC Architecture of Each Pad 130

7.3 Composition through Slot Connections 134

7.3.1 Distributed Versus Centralized Compositions 135

7.3.2 Update Dependency in Centralized Compositions 135

7.3.3 Update Dependency Architecture for Compositions 135

7.4 Compound-Document Architecture 136

7.5 Standard Messages between Pads 137

7.6 Physical and Logical Events and their Dispatching 140

7.6.1 Physical and Logical Events 141

7.6.2 Position Events and their Dispatching 141

7.6.3 User-Event Dispatching Mechanism 142

7.6.4 Geometrical-Operation Notification 146

7.7 Save and Exchange Format 146

7.8 Copy and Shared Copy 147

7.9 Global Variable Pads 149

7.10 Summary 149

References 150

8 Utilities for Meme Media 152

8.1 Generic Utility Functions as Pads 152

8.2 FieldPad for the Event Sharing 153

8.2.1 How to Share Events 154

8.2.2 FieldPad for Sharing Events 156

8.2.3 Manipulation of Event Information Data 160

8.2.4 Controllers over FieldPads 162

8.2.5 Conflict Resolution 164

8.2.6 Nested Shared Environments 164

8.2.7 Wormholes among Different Spaces 166

8.3 StagePad for Programming User Operations 166

8.3.1 An Outline of StagePad 167

8.3.2 Scripts and Casting 168

8.3.3 The Action Mechanism of StagePad 174

8.3.4 Dramas within Dramas 176

8.3.5 Dressing Rooms 178

8.3.6 Applications for Improving Pad Operability 180

8.4 Geometrical Management of Pads 181

8.5 Proxy Pads to Assimilate External Objects 181

8.6 Legacy Software Migration 185

8.7 Special Effect Techniques 186

8.7.1 The Clipping of a Pad 186

8.7.2 Alpha Channel 186

8.7.3 Zooming, Tilting, and Panning 187

8.7.4 Dissolution 187

8.8 Expression Pad 188

8.9 Transformation Pads 188

8.10 Summary 189

References 190

9 Multimedia Application Framework 191

9.1 Component Pads for Multimedia Application Frameworks 191

9.1.1 Text Processing Pads 192

9.1.2 Tables and Figures 192

9.1.3 Multimedia Pads 194

9.2 Articulation of Objects 195

9.2.1 Articulation of Multimedia Objects 195

9.2.2 Operations and Relations over Multimedia Objects 198

9.2.3 Application Linkage 198

9.3 Hypermedia Framework 199

9.4 Summary 204

References 204

10 IntelligentPad and Databases 205

10.1 Relational Databases, Object-Oriented Databases, and Instance Bases 205

10.2 Form Bases 208

10.2.1 Database Proxy Pads 208

10.2.2 Form Bases with a Single Form 209

10.2.3 Form Bases with Multiple Forms 210

10.2.4 Form Interface to Databases 211

10.2.5 QBE on Form Interface 214

10.3 Pads as Attribute Values 215

10.4 Multimedia Database 219

10.4.1 Articulation of Objects by Pads 219

10.4.2 Movie Databases 220

10.4.3 Articulated Objects in Media Objects 224

10.5 Hypermedia Database 224

10.5.1 Management of a Large Hypermedia Network 224

10.5.2 Hyperlinks as Queries 226

10.6 Geographical Information Databases 228

10.7 Content-Based Search and Context-Based Search 232

10.8 Management and Retrieval of Pads 234

10.8.1 Search for Pads with Partially Specified Composition Structure 234

10.8.2 The Encoding of View Composition Structures 238

10.9 Summary 239

References 240

11 Meme Pool Architectures 242

11.1 Pad Publication Repository and the WWW 242

11.2 Pad Publication and Pad Migration 244

11.3 Web Pages as Pad Catalog 245

11.4 URL-Anchor Pads 248

11.5 HTMLViewerPad with Embedded Arbitrary Composite Pads 250

11.6 New Publication Media 253

11.6.1 An Application to Scientific Publication 253

11.6.2 Publication and Reuse of Documents, Tools, Services, and Agents 257

11.7 Annotation on Web Pages 258

11.8 Piazza as a Meme Pool 260

11.9 Reediting and Redistributing Web Content as Meme Media Objects 263

11.9.1 Web Content as Memes 264

11.9.2 Application of Meme Media Technologies to Web Content 265

11.9.3 Related Research 266

11.9.4 XML and Pads 267

11.9.5 Extraction of an Arbitrary Web Content as Two- Dimensional Meme Media Objects 269

11.9.6 Direct Editing of HTML Views 272

11.9.7 Automatic Generation of Default Slots 275

11.9.8 Visual Definition of Slots for Extracted Web Content 279

11.9.9 Example Applications 280

11.9.10 Composition with More than One Wrapped Web Application 282

11.10 Redistribution and Publication of Meme Media Objects as Web Content 285

11.11 Summary 288

References 289

12 Electronic Commerce for Pads 291

12.1 Electronic Commerce 291

12.2 From Pay-per-Copy to Pay-per-Use 293

12.3 Digital Accounting, Billing, and Payment 294

12.4 Ecology of Pads in the Market 295

12.5 Superdistribution of Pads 297

12.6 Pad Integration and Package Business 301

12.7 Summary 303

References 304

13 Spatiotemporal Editing of Pads 305

13.1 Geometrical Arrangement of Pads 305

13.1.1 Tree Arrangement 305

13.1.2 Pad Cabinet Arrangement and Picture Index Arrangement 310

13.2 Time-Based Arrangement of Pads 311

13.3 Spatiotemporal Editing of Pads 315

13.3.1 Temporal Control of Geometrical Arrangement 316

13.3.2 Moving Pads 316

13.3.3 Hypermovie Framework 318

13.4 Information Visualization 320

13.5 Summary 323

References 324

14 Dynamic Interoperability of Pads and Workflow Modeling 325

14.1 Dynamic Interoperability of Pads Distributed across Networks 325

14.2 Extended Form-Flow System 331

14.2.1 Form-Flow Model in IntelligentPad 331

14.2.2 Virtual Forms to Assimilate Transaction-Based Systems 335

14.2.3 Form Generators and Form Annihilators 336

14.3 Pad-Flow Systems 337

14.4 Dynamic Interoperability across Networks 338

14.4.1 Network Publication of Form-Flow and Pad-Flow Systems 338

14.4.2 Bottom-up Integration across Networks 338

14.5 Workflow and Concurrent Engineering 339

14.5.1 Workflow Systems 340

14.5.2 Pad Flow as Workflow 342

14.5.3 Concurrent Engineering 343

14.5.4 Web Technologies and Workflow Systems 344

14.6 Summary 345

References 345

15 Agent Media 347

15.1 Three Different Meanings of Agents 347

15.2 Collaborative-and-Reactive Agents and Pads 348

15.3 Mobile Agents and Pads 351

15.4 Pad Migration and Script Languages 354

15.5 Summary 355

References 355

16 Software Engineering with IntelligentPad 357

16.1 IntelligentPad as Middleware 357

16.2 Concurrent Engineering in Software Development 359

16.3 Components and Their Integration 361

16.4 Patterns and Frameworks in IntelligentPad 363

16.4.1 Architectural Patterns, Design Patterns, Idioms, and Frameworks 363

16.4.2 Sample Composite Pads as Architectural Patterns 363

16.4.3 Pad Packages with Sample Compositions as Application Frameworks 365

16.4.4 Slot List as a Pattern 366

16.5 From Specifications to a Composite Pad 366

16.5.1 Use-Case Modeling 367

16.5.2 System Decomposition 368

16.5.3 From an Action Diagram to a Composite Pad 370

16.6 Pattern Specifications and the Reuse of Pads 373

16.6.1 Application Specification and Pattern Description for Primitive Pads 373

16.6.2 Pattern Description of Composite Pads 374

16.6.3 Composition and Decomposition of Patterns 377

16.6.4 Pattern Descriptions and the Reuse of Pads 379

16.6.5 An Example Development Process 381

16.7 IntelligentPad as a Software Development Framework 384

16.8 Summary 384

References 385

17 Other Applications of IntelligentPad 386

17.1 Capabilities Brought by the Implementation in IntelligentPad 387

17.2 Tool Integration Environments and Personal Information Management 387

17.3 Educational Applications 389

17.3.1 Teaching Japanese to Foreign Students 390

17.3.2 CAI in Physics and Mathematics 392

17.3.3 CAI in Control Theory 398

17.4 Web Page Authoring 400

17.5 Other Applications 401

17.5.1 CAD/CAM Applications 401

17.5.2 Financial Applications 403

17.5.3 Information Kiosk Systems 403

17.5.4 Electronic Libraries and Museums 403

17.5.5 Information Design Tools 405

17.6 Summary 406

18 3D Meme Media 407

18.1 3D Meme Media IntelligentBox 407

18.2 3D Application Systems 408

18.3 IntelligentBox Architecture 409

18.3.1 The Model-View-Controller Modeling 409

18.3.2 Parent–Child Relationship between Boxes 410

18.3.3 Message-Sending Protocol for Slot Connections 410

18.3.4 Shared Copies 410

18.4 Example Boxes and Utility Boxes 410

18.5 Animation with IntelligentBox 412

18.5.1 Motion Constraint Boxes 412

18.5.2 Shape Deformation Boxes 414

18.5.3 A RoomBox for Defining a 3D Shared Workspace 416

18.5.4 A CameraBox for the Interactive Viewing of a Box World 417

18.5.5 An Example Animation Composition 419

18.6 Information Visualization with IntelligentBox 419

18.6.1 Basic Functions for Interactive Information Visualization 420

18.6.2 3D Visualization and Virtual Materialization of a Single Retrieved Record 422

18.6.3 QBE Using a 3D Interface to a Database 425

18.6.4 Boxes as Attribute Values 426

18.6.5 Information Visualization and Virtual Materialization Framework Using IntelligentBox 427

18.6.6 3D Information Visualization of the WWW 436

18.7 Component-Based Framework for Database Reification 437

18.7.1 Flexible Definition of Visualization Schemes 437

18.7.2 Information Materialization through Query Composition 439

18.8 Virtual Scientific Laboratory Framework 445

18.9 3D Meme Media and a Worldwide Repository of Boxes as a Meme Pool 451

18.10 Summary 452

References 454

19 Organization and Access of Meme Media Objects 457

19.1 Organization and Access of Intellectual Resources 457

19.2 Topica Framework 459

19.3 The Application Horizon of the Topica Framework 462

19.4 Queries over the Web of Topica Documents 465

19.5 Related Research 468

19.6 Summary 471

References 472

20 IntelligentPad Consortium and Available Software 474

20.1 IntelligentPad Consortium 474

20.2 Available Software 475

20.3 Concluding Remarks 476

Author Index 477

Subject Index 479

About the Author 497