The Convergence of Language and Image: The Third Path to Semiotics Research

Semiotics, as a "Western science", sprouted in Western Europe and North America, and began as a modern manifestation full of Western colors. With the rise of Chinese semiotics, some skepticism followed, mainly focused on: in addition to F. Saussure. de Saussure) and C. Pierce Outside the path of Western semiotics represented by Peirce et al., Chinese semiotics is difficult to put forward its own theories with local colors. Is this really the case? The answer is no. We tentatively proposed the concept of "combined language and image" in an attempt to provide some new thinking for the exploration of the oriental path of world semiotics.

Western semiotics is characterized by the division of linguistics

Throughout the history of Western semiotics, the semiotic school represented by Saussure is dominated by its semantic centrism and normative tradition, and its adherence to the path is known as the first path of Western semiotics. The structuralist semiotics pioneered by Saussure reflect the idea of linguistics, which is actually a product of the Western pinyin alphabet culture, and can be interpreted as a binary and opposing model of phonological-image relationships: first, the separation of "language" and "elephant"; second, the realization of "symbols" through the suppression of "symbols". Saussure's linguistic semiotics inherits the symbolic cultural tradition of "divide and conquer" of pinyin: first, it insists on the distinction between linguistic symbols and non-verbal symbols (such as visual symbol elements); second, it treats nonverbal symbols in terms of formal language models (such as arbitrariness, combinatorial relations, and aggregation relations). Obviously, the divisional position of the pragmatic symbol concept inhibits the coding elements of nonverbal symbols, such as the image-likeness, intuition, and scene referentiality of physical symbols, etc., and enshrines "grammar" as the guideline, while "graph method" or "material method" is set aside.

Pierce semiotics, which arose almost simultaneously with Saussure, are often seen as the second path of Western semiotics (where the first and second do not deal with rankings of importance). Pierce distinguishes between image symbols, finger symbols, and symbols, representing images, objects (or indications, indexes of objects), and language, respectively. The biggest difference between Pierce and Saussure is the following four aspects. First, Pierce pays more attention to the dynamic meaning of symbols, rather than the systematic structural relationships of symbols. Second, Pierce pays attention to the substance of symbols, that is, the external related elements of symbols, such as the materiality of symbols, interpreters, objects of reference to symbols, and the rationality of symbols. Saussure's structuralist semiotics, on the other hand, was more concerned with formal rules of symbols, excluding the substantive elements of symbols. Third, Pierce's analytical model differs from Saussure's pragmatic model of verbal repression, in that he focuses on the distinction between the effects of expression between different heterogeneous symbols when referring to an object world: language to language, image to image, and object to object. In other words, Saussure is the division of language at the delayed level, and Pierce is the division of language at the distribution level. Fourth, Pierce approached heterogeneous symbols in an integrated rather than a cohesive way. In other words, he was concerned with the division within the family of synthetic symbols, while Saussure was concerned with the division of linguistic and nonverbal symbols.

The combination of language and image can be used as a new path of research

The concept of linguistic-image co-governance is relative to the Western concept of linguistic-image division, and it is also a symbolic meaning method that we have proposed after long-term thinking, which embodies a neutral symbolic concept. In the symbolic world, verbal symbols (including spoken and written languages, and various complementary alternatives to languages, such as braille dumb language, public signs, numeral signs, etc.) and symbolic symbols (including various visual symbols such as images, objects, rituals, and behaviors) are the most important two wings of the generalized symbol family. The so-called hezhi symbol refers to: "a symbol or symbolic expression unit that is internally implied and externally related to multiple heterogeneous symbolic elements (mainly linguistic images) and crosses and complements each other." "For example, modern and fashionable internet memes are the combined symbols of images and language symbols, which are neither images nor languages, but have the dual coding nature of semantics (words) and elephant symbols. Therefore, the heterogeneous symbol relationship handled by the "Hezhi Concept" is mainly the fusion relationship between semantics and symbols, referred to as "syntactic rule".

The combination of language and image is a common phenomenon of symbols. In the West, the phenomenon of the combination of grammatical symbols actually has a long history. As we all know, the important source of the Western alphabet is the ancient Egyptian holy book characters, which themselves are composed of pictorial pictograms and memorized consonant symbols, which is a typical symbol system of language and graph combination. For traditional Chinese culture, the combination of language and image is more typical. For example, our traditional chinese characters can often have both tangible (elephant symbols) and sound symbols (symbols) at the same time, and the ambiguity of the identity of the symbols actually reflects the cross-border integration of the oracle bone itself. For another example, the I Ching, which was written around the Western Zhou Dynasty, consists of two sets of symbols: one is the guayao symbol (象), which has the nature of a hieroglyphic character, and the other is the written symbol system that carries the meaning of the guayao symbol, that is, the guayan (language), which is compounded into a larger system of language and image combination.

We can distinguish between the combination of language and image as the combination of the heart shifting type and the middle type of combination of holding the middle type. "Empathetic harmony" refers to the cross-border of language and image while retaining the traces of cross-border or the tension of language and image differences, and people maintain a sense of pluralism in front of symbols. "Adhering to the middle-sized combination of governance" means that the language and image cross-border at the same time dissolve the rift of the cross-border, and the tension of the difference between the language and the image is replaced by a sense of unity. The former, such as the hieroglyphs of the ancient Egyptian holy book body, which are formally marked when converting language images: for example, when used as a definite symbol (symbol), it is always at the end of the word, and when used as a musical note (symbol), it is placed before the definite symbol, and the holy book word maintains a cross-border trace that is, a mark during the process of language conversion. Relatively speaking, the oracle bone is a combination of medium-sized language symbols, and the semantics (false borrowed or shape-sound characters) and the symbols (pictograms or shape-sound characters) hide the traces of the cross-border of language and image in the language-image conversion, and the medium-sized ancient Chinese characters represent the rational semiotic tradition of Eastern symbols; the center-shifting ancient Egyptian scripts characterize the Western semiotic tradition of de-rationalization and arbitrary statute-dominated since ancient Egypt.

The yuxiangzhi medium-sized view of combined governance can be derived from the Confucian classics. The most concise and generalized expression of the Confucian middle way is "clinging to its two ends and using it for the people", referred to as "holding on to the two ends", that is, dissolving the polar opposition between the two ends and adopting an impartial position of holding on to the middle. The way of the middle way is embodied in the Yi Chuan as a symbolic idea of adhering to the combination of medium-sized language and image: the meaning is exhausted, and the image is written in words. Therefore, the speaker is like the image, and he forgets the word; the elephant, so he has the intention, and he is proud and forgets the image. The tradition of clinging to medium-sized language and imagery makes Chinese inclined to a holistic and construal thinking: reading and interpreting when facing visual images, and taking a visual and perceptible approach to grasping abstract concepts. The disappearance of tension and distance between language and image has the characteristics of the construction of generative symbols: the emphasis is on the application of the harmony of language and image rather than on the meta-symbol reflection of the language-image relationship itself.

Contemporary Western also has many expositions on the idea of the combination of language and image, but basically describes the relationship between language and image from the perspective of shifting the heart. Derrida called the movement of tension between the primitive script or the transmutation of the symbols and symbols included in the original script "heterogeneity." R. Barthes Barthes), M. Foucault Foucault), J. Baudrillard Baudrillard), J. Lyotard), W. Michelle), Lotman (Ю.Лотман) and others have made in-depth explorations of the relationship between words and images, such as the relationship between words and diagrams, and the transfer of hearts and minds between words and things. The American sinologist E. Ferno Rossa Fenollose) calls the mid-sized way of writing the medium-sized xianghezhi "ideographic thinking".

Construct philological semiotics

Adopting the way of "combining language and image", the establishment of China's semiotic methodology is not to single-handedly promote the oriental symbolic thinking in the language and image, otherwise it is closed doors and self-imposed. With an inclusive mindset, we should build a dialogue-based and mutual learning relationship between the Middle-Class of the East and the Shifting Mind of the West.

Our view of "combined governance" contains two meanings: first, it is a contrast term relative to "division and rule", which is the dissolution of the division and opposition of heterogeneous symbols (mainly referring to linguistic images); second, it is a symbolic way, mainly including two types of shifting of the heart and clinging to the middle. The symbolic connotation of "language-image co-governance" composed of these two points not only contains the semiotic essence of Western cross-media, multimedia, multilingual, and linguistic narrative, but also absorbs the ancient Chinese tradition of "neutralization", "imagery" and "six books", and then forms a semiotic paradigm with Oriental characteristics - semiotics of linguistic and symbolic co-governance.

(Author Affilications:Research Center for Applied Communication of Language Symbols, Tianjin University of Foreign Chinese; School of Literature and Journalism and Communication, Ocean University of China)

Edit: Zong Yue