Skip to main content

A non-photorealistic rendering method based on Chinese ink and wash painting style for 3D mountain models

Abstract

The ink and wash painting stylized rendering with oriental aesthetic features is an important topic in non-photorealistic rendering (NPR). The research on the stylized rendering of two-dimensional (2D) models has received extensive attention, but there is still a lack of systematic research on the three-dimensional (3D) rendering effects of different brush strokes and texture mapping methods. In this paper, we mainly render the 3D mountain models from the whole ink and wash painting rendering process, different winkle rendering methods, and the canvas texture based on spatiotemporal consistency. First, the proposed rendering process is divided into feature lines rendering and interior area stylization, and the wrinkle rendering is carried out by the method of texture synthesis and texture mapping. In addition, mesh parameterization is introduced to avoid stretching and distortion that may occur during the texture mapping. Then, an empirical model is used to simulate the ink diffusion effect in the rendered image space. Finally, we investigate a dynamic canvas texture method considering temporal coherence, which uses a noise-based algorithm to generate the canvas texture that maintains the 2D appearance of the canvas under camera motion. Experimental results show that the winkle rendering and texture mapping method proposed in this paper adds rich details to 3D mountain models, and the “Shower Door” effect does not occur in the case of camera motion.

Introduction

The art stylized rendering technology based on non-photorealistic rendering (NPR) depicts the objective world by imitating the painters’ perspective. It does not pursue visual effects similar to photography, nor strictly follows the optical mapping rules. This technology has achieved good artistic effect in the simulation of watercolor painting, oil painting, pen painting and other painting styles [1,2,3]. As one of the carriers of Chinese traditional culture, ink and wash painting has its unique artistic charm. A comprehensive study of the ink style rendering method of three-dimensional (3D) mountain models can not only reduce the burden of background painting for the production of ink and wash painting works, but also enrich the perspective changes of ink and wash painting works, which is of great significance for the inheritance and development of ink painting [4,5,6].

Landscape painting is one of the three major themes of Chinese ink and wash painting, and there are many excellent works. It is of great significance to learn the characteristics of these famous landscape paintings and carry out the 3D stylized rendering in cultural inheritance and practical application [7, 8]. On the one hand, ink and wash painting carries the aesthetic conception and artistic expression of Chinese style. Converting images or 3D models into ink and wash painting through ink stylized rendering, or creating ink and wash painting works through interactive rendering, can better show the artistic charm of ink and wash painting [7]. The British Museum has dynamically presented the Ming Dynasty painting “Reading in the Autumn Forest” in a 3D form. Starting from the 2D original work, it leads the audience to enjoy the scenery in the painting from various angles as the camera changes [8]. This way of ink style display based on 3D model has the advantages of spatial sense and expressive force. Therefore, relevant research is of great significance for the inheritance and development of ink and wash painting.

On the other hand, the study of ink stylized rendering method can improve the production efficiency and promote the practical application of ink and wash painting works. In traditional ink animation, if the background of mountains changes, it is necessary to redraw the background of ink and wash painting by hand [9]. This traditional production method is very labor-intensive and time-consuming. If we can realize the stylized rendering of ink and wash painting for the 3D mountain models, it can not only reduce the burden of background rendering for the production of ink and wash painting works, but also bring more creation space. With the development of technology, stylized rendering of ink and wash painting has been gradually applied to game production, film and television animation, cultural exhibition and other fields [10].

Ink and wash landscape painting has unique artistic characteristics. Its brushwork is changeable, and the shading effect is very different on different materials such as silk and rice paper. The different concentration of the ink also makes ink paintings have rich variations between black and white. Due to the limitation of style transfer in the feature extraction of Chinese ink and wash paintings in multiple painting styles, the research on the ink stylized rendering of 3D mountains and rocks is still carried out in the framework of processing feature lines and interior areas separately.

The traditional ink painting background processing generally adopts the methods of superimposing static rice paper texture on the result image of ink and wash rendering of 3D model. Although these methods have a visual appearance similar to the texture of the rice paper, they are only suitable for the rendering of static frames. The "Shower Door" effect occurs when the camera moves relative to the model [11]. To ensure the spatiotemporal consistency of NPR, some researchers have proposed and applied the primitive-based method and texture-based method to the relevant work. However, these works mainly simulate the texture of paper, and require more computing resources [12], which is difficult to calculate in real time on ordinary computers.

To address these challenges, we propose an overall ink and wash stylized rendering method which applies a variety of related algorithms to optimize the rendering effect. According to the traditional regular methods, the proposed rendering method is divided into feature lines rendering and interior area stylization. In addition, mesh parameterization is introduced to avoid stretching and distortion that may occur during the texture mapping. Then, an empirical model is used to simulate the ink diffusion effect in the rendered image space, and a dynamic canvas texture method considering temporal coherence is investigated to generate the canvas texture that maintains the 2D appearance of the canvas under camera motion. The main contributions of this paper are as follows:

  1. 1.

    The rendering method of the surface and interior area of 3D mountain models is improved and texturing method is adopted to make wrinkle rendering by combining texture synthesis with texture mapping. The wrinkle texture samples in ink landscape painting are firstly collected for preprocessing, and then the processed texture samples are synthesized to get high resolution wrinkle texture. Finally, the feature line rendering results based on the curvature of the mountain models are used to control the texture mapping. Compared with the existing texture rendering methods based on the brush-style model, the proposed method has a significant improvement in real-time performance.

  2. 2.

    The mesh parameterization method is used to avoid the stretching and distortion that may occur in the later texture mapping. Compared with the existing texture-based methods, the proposed method can reduce the texture stretch and obtain a visual effect closer to that of real ink works.

  3. 3.

    Based on geometric shader, we build a virtual canvas that moves with the objects. Algorithms based on Perlin Noise and Gabor Noise are used to generate simulated rice paper and silk canvas textures, which can better maintain the 2D appearance of the canvas when the camera is moving. The proposed method can ensure the spatio-temporal consistency of canvas texture well and avoid the obvious “Shower Door” effect.

Related work

The ink and wash painting stylized rendering method can generally be divided into three aspects: ink stylized rendering of images, ink interactive rendering and ink stylized rendering of 3D models. Next, the related work will be described from these three aspects.

Ink stylized rendering of images

In the ink stylized rendering of images, the real landscape image is generally taken as the input of the model, and the image processing algorithm can be used to process the input image to output the image with ink painting style. Using the corresponding image processing algorithm, the frame of ink and wash painting can be synthesized. Basic stroke textures are then collected from hand-drawn ink and wash painting to form a collection of stroke textures. This collection of stroke textures and drawn control image frame can be used to synthesize ink and wash painting [13]. Similarly, a rendering algorithm based on a physical model is proposed to simulate the effect of an ink spread on an absorbent sheet of paper and can convert a digital image into a color ink painting [14]. In addition, through the process of image corrosion, anisotropic diffusion, extraction and superposition of edge information, static real scene images such as landscapes and buildings can be automatically converted into ink style images [15]. With the development of deep learning algorithms, the application of these intelligent algorithms to the rendering of ink and wash painting has attracted the attention of researchers. By applying the art style algorithm of convolutional neural network to the creation of ink and wash painting, any image can be rendered in ink and wash painting style [16].

Ink interactive rendering

Ink interactive rendering focuses on how to simulate traditional ink and wash painting media to realize the digitalization of drawing process [17]. Early researchers have proposed such interactive digital rendering systems, which use a local equilibrium model to simulate the dispersion of water and ink on paper [18]. To make ink interactive rendering more flexible, some researchers model the writing brush, rice paper and ink respectively. Users can use this method to create realistic ink paintings in the computer [17]. Furthermore, some researchers build a model that contains a skeleton and a flexible curved surface, and propose an interactive brush painting system. The system simulates the stretching of the brush tip on the paper surface during the interaction process and the shape of the brush tip when rubbed against the paper surface. This physics-based method allows realistic simulations of brush calligraphy and Chinese ink and wash painting [19]. On the other hand, a parameterized physical brush model based on experience has been proposed to simulate the traditional creation process of Chinese calligraphy and ink painting. This model only considers the changes in the contact between the brush and the paper surface and does not attempt to discuss the subtle changes in the brush from the point of view of a real physical model. The experimental results show that this model can simulate the diffusion effect of ink with a relatively fast calculation speed [20].

Ink stylized rendering of 3D models

The main research contents of ink stylized rendering of 3D models usually include extraction and rendering of contour lines, ink stylization of interior areas, and ink diffusion simulation [21, 22].

A basic framework of ink stylized rendering based on 3D models is used to deal with the contour lines and interior areas respectively. This framework is adopted by many subsequent ink stylized rendering studies for 3D models [21]. Some researchers use geometric shaders to extract contour lines, widen the contour lines, and then obtain ink stylized rendering results through texture mapping and edge particle [22]. In addition, the diffuse illumination value can be quantified into a specific gray value, which can obtain different coloring effects of the interior areas. The gray texture is superimposed on the basis of the Gaussian smooth segmentation of the diffuse illumination value, and the gray value obtained by sampling on the gray texture is used as the corrosion coefficient, which can effectively control the color change of the interior areas of the model. This method has achieved a good coloring effect on the models of houses' exterior walls in Chinese ink and wash painting [23].

The simulation of ink diffusion effect usually starts from the visual effect of ink diffusion, and does not seek to reproduce the real interaction between ink and paper. The conventional simulation method uses the image filtering algorithm to process the results of line rendering and interior coloring [24]. A fractal-based algorithm is used to simulate the irregular contour lines after ink diffusion. This algorithm uses a fluid simulation model to simulate ink diffusion, and gets a good simulation effect [25]. However, the calculation of this fluid model is more complicated. In addition, an empirical diffusion model is put forward to divide ink painting into original stroke area, main diffusion area and edge area. The synthesis speed of this model is fast, and the diffusion simulation of ink painting and 3D rendering images can get good visual effect [26].

System model and algorithms

In order to realize the ink and wash stylized rendering of the 3D mountain model, we divide the rendering process into feature lines rendering and interior areas shading. For lines rendering, the method based on curvature estimation is used. The rendering results include not only the external contour lines but also the internal lines, so they are collectively called feature lines. For the interior areas rendering, the wrinkle rendering is added by using texture synthesis and texture mapping. After the rendering of the lines and interior areas of the model, the dynamic canvas texture with consideration of temporal coherence is used to realize the dynamic simulation of silk and rice paper texture, and finally the ink diffusion of the rendered image is simulated.

The algorithm framework for ink and wash rendering proposed in this paper is shown in Fig. 1.

Fig. 1
figure 1

3D mountain model ink painting stylized rendering algorithm framework

Curvature based feature line rendering

In this paper, the pre-integrated skin shading method [27] is used to calculate the curvature value \({k}_{i}\). The points with larger curvature value are the points of the feature line. According to the characteristics of ink and wash stylized rendering, the color near the contour line is close to black, and the interior flat area is close to white. In addition, to control the color range of the contour, the parameter \({R}_{CurveFactor}\) can be set by the user. Based on this, the initial color value of the characteristic line at point \(i\) is calculated as:

$$ col_{i} = 1 - k_{i} * R_{CurveFactor} $$
(1)

Then, the threshold value \(\_Curv0\) is used to binarize \({col}_{i}\). The part less than the threshold value is rendered to the specified feature line color \(\_LineColor\), and the part above the threshold is rendered to the specified background color \(\_BgColor\). The rendering results and parameters of the curvature-based feature line are shown in Fig. 2, where \(\_LineColor\) is represented by red, green and blue (RGB) components in the range of 0 ~ 1.

Fig. 2
figure 2

Rendering different results and parameters based on curvature-based feature lines

Figure 3 shows the result of feature line rendering of a large mountain rock model modeled after the part of the famous Chinese painting “Autumn Mountain Asking the Road”, where Fig. 3a is the result of directly coloring the model using the initial color value, and Fig. 3b is the result of final feature lines rendering.

Fig. 3
figure 3

Feature line rendering results for large mountain models

Basic internal coloring

According to the traditional interior coloring methods, ink and wash paintings have the principle of “focus, thick, heavy, light, and clear” to clarify the ink color into five colors. To improve on the traditional internal coloring algorithm at multiple levels, we first calculate the diffuse reflection coefficient \({t}_{d}\) of the model surface. Based on the feature line rendering, the experiment of directly quantifying \({t}_{d}\) to simulate internal coloring is carried out. The quantization formula of \({t}_{d}\) is shown in Eq. (2), where \({t}_{1}\sim {t}_{4}\) and \({C}_{1}{\sim C}_{5}\) are adjustable gray value parameters ranging from 0 ~ 1, and the quantized result is \({C}_{d}\).

$$ C_{d} = \left\{ {\begin{array}{*{20}c} {C_{1} } & {\,\,if\mathop t\nolimits_{d} \le \mathop t\nolimits_{1} } \\ {C_{2} } & {\,\,\,\,\,\,\,\,\,\,\,\,\,\,if\mathop t\nolimits_{1} < \mathop t\nolimits_{d} \le \mathop t\nolimits_{2} } \\ {C_{3} } & {\,\,\,\,\,\,\,\,\,\,\,\,\,\,if\mathop t\nolimits_{2} < \mathop t\nolimits_{d} \le \mathop t\nolimits_{3} } \\ {C_{4} } & {\,\,if\mathop t\nolimits_{d} > \mathop t\nolimits_{4} } \\ {C_{5} } & {\,\,if\mathop t\nolimits_{d} > \mathop t\nolimits_{4} } \\ \end{array} } \right. $$
(2)

To simulate the irregular edge shape of ink painting, a corrosion coefficient \({t}_{w}\) is introduced. The corrosion coefficient \({t}_{w}\) is the color value sampled on a random gray texture and converted to gray value, and it can be calculated by mixing \({t}_{w}\) and \({t}_{d}\) as follow equation:

$$ t_{dw} = clamp(t_{d} + t_{w} + 1) $$
(3)

where clamp is a truncation function that only retains values in the range of 0 to 1. The corrosion coefficient \({t}_{w}\) is sampled from the grayscale texture shown in Fig. 4a. It should be noted that this image has been enlarged 100 times and taken a screenshot of part of it to show the details. The noise texture image used in our method is shown in Fig. 4b which also has been enlarged 100 times, and then the experimental result of quantifying \({t}_{dw}\) is shown in Fig. 4c.

Fig. 4
figure 4

Different experimental results of internal coloring

Direct quantification can lead to very serious color scale jumps. To smooth the result to get a more natural effect, we use a Gaussian ambiguity function to handle the transition aera. We apply a one-dimensional Gaussian blur to adjust the color of the output. The shading effect after Gaussian blur is shown in Fig. 4d. To make the color transition more natural, the operation of superimposed noise is carried out before quantization. The final experimental result is shown in Fig. 4e.

To provide richer coloring changes of the interior aera of the model, the initial color value \({col}_{i}\) of the feature line at point \(i\) is directly used to calculate the interior region of the model. \({col}_{i}\) is multiplied by \({t}_{dw}\) before adding noise, and then the Gaussian smoothing ladder is carried out. After stacking the static paper texture, the experimental result is shown in Fig. 4f.

Wrinkle rendering

Based on traditional interior shading methods, the wrinkle rendering is added by using texture synthesis and texture mapping. First of all, texture samples are obtained from the ink landscape painting. After preprocessing, large-scale texture is obtained by texture synthesis. Then, the interior region of the mountain model is obtained by the feature lines. Finally, the large-scale texture is mapped to the 3D mountain model. In addition, before the texture mapping, the mesh parameterization is used to avoid the stretching and distortion that may occur during the texture mapping.

Mesh parameterization

To reduce the stretching deformation in texture mapping, we use the CGAL library [28] for mesh parameterization. Early parameterization method can guarantee one-to-one mapping when the boundary is a convex polygon, but the deformation is serious [29]. Later, a mean parameterization method with fixed boundary is proposed to realize the discrete approximation of conformal mapping [30]. In addition, a plane parameterization method can achieve a good balance between conformal mappings, but it cannot guarantee local one-to-one mapping [31]. Therefore, mesh parameterization algorithm is used in our method [30], and Fig. 5 shows the effect comparison of checkerboard mapping.

Fig. 5
figure 5

Comparison of checkerboard texture effect before and after mesh parameterization

Texture sample preprocessing and winkle texture synthesis

As shown in Fig. 6a, the brushwork is often not clear enough due to the texture sample image collected directly from the original image. Therefore, the preprocessing of contrast enhancement is performed on the texture sample image. The method of linear grayscale transformation is adopted to adjust the grayscale of the original texture samples. It is assumed that the grayscale range of interest in the original image \(f(x,y)\) is \([a,b]\), the grayscale range of transformed image \(g(x,y)\) is \([c,d],\) and the transformation formula is as follows:

$$ g(x,y) = \left\{ {\begin{array}{*{20}c} c \\ {\frac{{d - c}}{{b - a}}[f(x,y) - a] + c} \\ d \\ \end{array} } \right.\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\begin{array}{*{20}c} {\,if\user2{ }\,f(x,y) \le a} \\ {\,\,\,\,\,\,\,\,\,\,\,if\user2{ }\,a < \user2{ }f(x,y) \le b} \\ {if\user2{ }\,f(x,y) > b} \\ \end{array} $$
(4)
Fig. 6
figure 6

Texture sample preprocessing and winkle texture synthesis

The texture samples collected from the ink and wash works need to be pretreated, and the results before and after contrast enhancement are shown in Fig. 6a and b. Texture synthesis combined with texture mapping is adopted to render 3D mountain model to achieve the effect of rich details inside the texture. Taking the contrast-enhanced texture samples in Fig. 6b as input, we use image quilting algorithm and graph cut texture synthesis algorithm [32] to obtain large-sized textures as shown in Fig. 6. In this paper, a more natural image quilting algorithm is adopted.

Basic internal coloring

After large-size winkle texturing, we obtain the winkle texturing color value corresponding to the vertex of the 3D mountain model by sampling the texture at the satisfying position. Texture coordinates used in texture sampling are the one after mesh parameterization and adjustment. It is important to note that texture mapping is performed only when the result of feature line rendering is \(\_BgColor\).

The internal coloring of the mountain model is combined with the basic internal coloring algorithm and texturing color. Equation (5) is used to calculate the corrosion coefficient \({t}_{w}\) in the basic internal coloring, where \({W}_{wrinkleTexcol}\) is the texture color value of texturing sampling, and \({R}_{InnerTexFactor}\) is the mixing coefficient of texturing color value. By adjusting this coefficient, texturing color can be better integrated with surrounding colors.

$$ \mathop t\nolimits_{w} = dot(\mathop W\nolimits_{wrinkleTexcol} .rgb,(0.3,0.59,0.11))/\mathop R\nolimits_{InnerTexFactor} $$
(5)

After adding the winkle rendering, the results of the stylized rendering of the ink are shown in Fig. 7. Figure 7a–c show the experimental results of the linen wrinkle, raindrop wrinkle, and chopping and chowing wrinkle respectively. It can be observed from the figures that they all have the style of ink painting, and there are obvious differences in expression techniques.

Fig. 7
figure 7

Experimental results of wrinkle texturing rendering

Diffusion simulation

After the feature lines rendering and interior shading, the scene is rendered into a render target texture. We use the empirical ink diffusion simulation model to process it, and use the background texture generated by the rendering background texture section to determine the absorption capacity of the canvas. The ink and wash strokes are divided into original stroke region, main diffusion region, and boundary diffusion region [33].

The main diffusion region is generated by expanding the boundary from the original boundary one pixel at a time, and the times of diffusion are determined by the diffusion distance\(d\). The transparency of the new pixel is \({A}_{p}\), and its calculation formula is as follows:

$$ A_{p} = 1 - \frac{1}{k}\sum\limits_{i = 1}^{k} {(A_{i} * f} ) $$
(6)

where \({A}_{i}\) is the transparency of the pixels located on the boundary formed by the previous enlargement in the four neighborhoods of the point \(p\), and \(f\) is a color attenuation parameter that can be adjusted by the user.

The generation of boundary diffusion region starts from the boundary pixels of the main diffusion region. Each pixel is a basic unit that absorbs ink, and each point simultaneously absorbs ink in proportion from its eight connected neighborhoods. Assuming that the maximum absorbable ink amount of the current pixel is \({I}_{p}\), the initial amount of ink at the boundary point \(i\) of its eight neighborhoods is \({I}_{i}\), the remaining amount of ink before absorption is \({I}_{i}^{^{\prime}}\), and a residual ink value \({I}_{r}\) is specified in advance. When the ink content \({I}_{i}^{^{\prime}}\) of the boundary point is not larger than \({I}_{r}\), the boundary point does not provide ink.

Assuming that the entire absorption process is completed through \(n\) iterations. When \({I}_{i}^{^{\prime}}>{I}_{r}\), \(\Delta {I}_{i}\) refers to the amount of ink absorbed from the adjacent boundary point \(i\) each time, and it is calculated as the following equation.

$$ \Delta I_{i} = (I_{p} /n) * R_{ink} (I_{i} ) $$
(7)

where \({R}_{ink}({I}_{i})\) is a function for calculating the absorptivity of the boundary point \(i\), suppose there are a total of m boundary points in the eight neighborhoods of the current pixel, the absorptivity at the boundary point \(i\) is as follows:

$$ R_{ink} (I_{i} ) = (I_{i} - I_{r} )/\sum\limits_{k = 1}^{m} {(I_{k} - I_{r} )} $$
(8)

where the value of \({I}_{r}\) affects the shape of the boundary. The initial ink content \({I}_{i}\) at the boundary point of the basic stroke is equal to the pixel color value at that point [28]. We achieve final result of ink diffusing with different parameters in Fig. 8. By comparing Fig. 8a and b, it can be observed that significantly different effects can be achieved by adjusting the values of some parameters. The experiments complete the separate rendering of the lines and the interior area, and bring a lot of details for the winkle rendering of the interior area on the surface of mountain models. Moreover, the method simulates the ink diffusion of the entire rendering results, and realizes the simulation of the ink painting effect.

Fig. 8
figure 8

Stylized rendering results of different mountain models and parameters of ink diffusion simulation stage

Canvas texture based on spatiotemporal consistency

Canvas texture is an important element of the ink and wash style, which reflects the main characteristics of the painting medium in the appearance of the texture. In this paper, a canvas that moves with the object is constructed based on the geometry shader. The noise-based method is used to generate the canvas texture, which maintains the 2D appearance of the canvas under the camera motion. Comparing to the traditional processing of ink and wash canvas, the distinct “Shower Door” effect can be avoided in our method, and the motion coherence and the flatness of the canvas can be balanced better to meet the real-time target.

Rendering method based on Gabor Noise

The Gabor Noise rendering method based on sparse convolution and Gabor kernel can accurately control the spectrum of noise through such parameters as phase shift, dominant frequency and bandwidth [34]. Gabor kernel can be calculated by the product of Gaussian envelope and a harmonic, as shown in Eq. (9).

$$g\left(x,y\right)=K{e}^{-\pi {a}^{2}({x}^{2}+{y}^{2})}\mathit{cos}[2\pi {F}_{0}(x\mathit{cos}{\omega }_{0}+y\mathit{sin}{\omega }_{0})]$$
(9)

where K and a represent the amplitude and width of the Gaussian envelope respectively, \({(F}_{0},{\omega }_{0})\) represents the frequency of harmonic in polar coordinate system, \({F}_{0}\) and \({\omega }_{0}\) represent the amplitude and phase shift, respectively. Gabor Noise can be expressed as the sum of a series of Gabor kernels with random weights and random positions, which can be calculated with the following equation:

$$n\left(x,y\right)=\sum_{i}{\omega }_{i}g({a}_{i},{F}_{0i},{\omega }_{0i};x-{x}_{i},y-{y}_{i})$$
(10)

where i represents the number of Gabor kernels, \({\omega }_{i}\) represents the random weight, {(\({x}_{i}\),\({y}_{i}\))} represents the random position. The difference of the Gabor kernels is determined by bandwidth a, frequency \({F}_{0}\) and direction\({\omega }_{0}\). Therefore, different kinds of noises can be obtained by using different parameters a, \({F}_{0}\) and\({\omega }_{0}\). The Gabor kernel radius r is determined by bandwidth a, which can be calculated with the following equation:

$$ r = {\raise0.7ex\hbox{${\sqrt {\frac{{\log 0.05}}{\pi }} }$} \!\mathord{\left/ {\vphantom {{\sqrt {\frac{{\log 0.05}}{\pi }} } a}}\right.\kern-\nulldelimiterspace} \!\lower0.7ex\hbox{$a$}} $$
(11)

The concept of grid is introduced in the calculation of 2D Gabor Noise, and the pseudo random number generator (PRNG) is used to calculate the value of each cell’s Gabor kernel. The size of the cell is equal to the radius r of the Gabor kernel. For a specific pixel, the \(3\times 3\) neighborhood around it will be used as a cell to calculate the noise. The noise calculation process is equivalent to the accumulation of multiple Gabor kernels. The number of kernels follows the Poisson distribution with the mean value of N′, which is an adjustable parameter, usually between 16 and 128. The coordinates {(\({x}_{i}\),\({y}_{i}\))} and weight \({\omega }_{i}\) of the kernel are random numbers in the range of 0 ~ 1.

In addition, the NPR Gabor Noise calculation method is improved on the basis of the Gabor Noise calculation. In NPR Gabor Noise, the noise gray value is calculated in 2D screen space, while the point distribution is calculated in 3D space. In this way, we can not only maintain the 2D characteristics of the texture, but also track the motion of 3D objects.

Dynamic canvas construction based on geometry shader

The dynamic canvas construction algorithm in this paper is based on geometric shaders. In each input triangle patch, the number of small squares to be generated is calculated according to the area of triangles in screen space. The position of the center point of the small square is generated by the method of barycentric coordinates. The side length of the small square is twice the radius r in Gabor Noise. This paper takes each small square as the basic unit to construct noise texture, so the four vertices of each little square are given texture coordinates for (0,0), (0,1), (1,0), and (1,1). The gray value of the noise is calculated using the texture coordinates as input in the slice shader.

In the proposed method, the calculation method of the distribution of points by the NPR Gabor Noise is adopted. For each visible triangle, the number of small squares in the triangle should be determined first. The calculation method is as follows:

$$M={2}^{[{\mathit{log}}_{2}N]}$$
(12)
$$N=\lambda A$$
(13)

where \(A\) is the projected area of the triangle in the screen space, \(\lambda \) is calculated according to Eq. (14), and \(r\) is calculated according to Eq. (11).

$$\uplambda =\frac{{N}^{^{\prime}}}{\pi {r}^{2}}$$
(14)

For the calculation of the location of the center point of each small square, the pseudo random number generator [34] is used to generate two random numbers \(s\) and \(t\) between (0, 1). According to \(s\), \(t\) and Eq. (13), we can obtain the barycenter coordinate coefficients \(a\), \(b\) and \(c\) by Eq. (15)–(17). By multiplying the model coordinates of the three vertices of the triangle by \(a\), \(b\) and \(c\), the coordinates of each center point in the model space can be obtained.

$$a = 1 - \sqrt{t}$$
(15)
$$b = (1 - s)*\sqrt{t}$$
(16)
$$c = s*\sqrt{t}$$
(17)

It should be noted that the seed initially used by the pseudo-random number generator is also the initial seed for obtaining the Gabor Noise value later. The seed is obtained according to model space coordinates of triangle vertices to ensure that each triangle uses the same seed, so as to ensure time continuity in dynamic situation and reduce flicker.

After obtaining the coordinates of the center point of the small square, a small square is generated with this point as the center and \(r\) as half of the side length in the tangent plane of the point. In addition, we need to specify texture coordinates for the four vertices of the generated small square according to (0, 0), (0, 1), (1, 0), and (1, 1).

After the transformation of the pixel is completed in the geometry shader, the texture coordinates of the vertex of the pixel are used as input. We use Gabor Noise or Perlin Noise algorithm to get the noise gray value of corresponding point, respectively to realize the dynamic simulation of silk and rice paper. For the calculation of Gabor Noise, since the generated texture result has obvious directionality, the method of mixing the two noise calculation results is used. When calculating the grayscale value of noise in the second time, the texture coordinates are exchanged to get the texture results of horizontal and vertical interweaving.

Experimental results and analysis

To verify the effectiveness of the above method, we realize the main functions through programming, and carry out a lot of analysis and verification of the experimental results. The main experimental environment and corresponding parameters adopted in this paper are shown in Table 1.

Table 1 Lab environment

The winkle rendering of mountain model with different texturing methods

In the process of the development of landscape painting, there are many different methods of texturing. In this paper, different types of texture samples are used for experiments, and the rendering results in Fig. 9 are obtained.

Fig. 9
figure 9

The wrinkle texturing experiment to simulate real mountain model painting

The rendering speed of the algorithm in this chapter is shown in Table 2. The parameter n in the table is the number of iterations in the edge diffusion region simulation. It can be seen that the parameter n has a great impact on the real-time performance of the whole ink style rendering.

Table 2 This chapter algorithm rendering speed

Experimental results of ink background canvas texture

For the dynamic canvas construction algorithm based on the geometric shader, this paper firstly uses the texture coordinates as the input of the noise value, and takes the plane mesh as the background model. The irregular black spots on the mountain model represent the impurity position on the canvas. When impurities follow the model, the canvas follows the model. Compared with the traditional superimposed static rice paper in Fig. 10a, which has obvious “Shower Door” effect, the 2D texture appearance is better maintained while ensuring the movement consistency. The experimental results of dynamic canvas construction based on geometric shaders are shown in Fig. 10b.

Fig. 10
figure 10

Static and dynamic rice paper texture rendering results at different camera positions

Overall rendering and evaluation

Using the overall rendering method proposed in this paper, a variety of different mountain models are rendered in ink and wash painting style as shown in Fig. 11a–c. It can be observed from the figure that the rendered mountain models have obvious characteristics of Chinese ink and wash painting.

Fig. 11
figure 11

Experimental results of ink and wash stylized rendering of 3D mountain models

To enrich the content of the screen, the famous “Akiyama asked map” is selected for local simulation. It should be noted that the simulation focuses on the trend of the mountain rather than the strict reproduction of the work. In addition, considering the richness of the screen, trees, people, pavilions and other elements are added to the 3D scene. Because the proportion of these elements in the picture is very small, only the method based on vertex vector and line of sight vector dot product is used for line rendering. The rendering result of the initial view angle is shown in Fig. 11d.

Conclusions

In this paper, 3D mountain models are taken as the research object. The wrinkle rendering of the mountain models and the canvas texture based on spatiotemporal consistency are studied, and the complete ink and wash rendering process is realized. In addition, the algorithm of this paper still maintains the 2D appearance of the ink background canvas in the case of camera motion, and avoids the obvious “Shower Door” effect. Finally, the real-time performance achieves good results in the entire process of ink and wash rendering the 3D mountain model. Since the proposed method creates the corresponding 3D rendering on the basis of learning the characteristics of ink painting, it can only be well applied to the rendering of Chinese ink and wash painting, but cannot be applied to other types of paintings, such as figure painting. In future work, we will look into different types of paintings and hope to design more general 3D rendering methods.

The rendering method proposed in this paper can better ensure the spatiotemporal consistency of the canvas texture, and meet the real-time requirements of computer operations. The experimental results show that the rendered effect conforms to the texture characteristics of the ink canvas. This method can not only be used for 3D reconstruction of Chinese ink and wash painting to facilitate the inheritance of cultural heritage, but also can be used in game production, film animation, cultural exhibition and other applications. With the continuous development of technology, especially the application of deep neural network, 3D rendering technology will be widely used. Since traditional paintings are created by painters with specific painting skills in a large number of practices, they often have high artistic characteristics. Although computer painting can achieve a very realistic effect by learning the characteristics of traditional painting, and is convenient for mass communication and applications, computer painting is not able to create works with profound artistic value like painting masters at the present stage. Therefore, it also brings new challenges for 3D art stylized rendering and computer painting technology in the future.

Availability of data and materials

All data for analysis in this study are included within the paper.

Abbreviations

3D:

Three-dimensional

2D:

Two-dimensional

NPR:

Non-photorealistic rendering

GPU:

Graphic processing unit

RGB:

Red, green and blue

PRNG:

Pseudo random number generator

References

  1. Liu Y, Li R, Yang J, et al. China and the steppe: technological study of precious metalwork from Xigoupan Tomb 2 (4th–3rd c. BCE) in the Ordos region Inner Mongolia. Herit Sci. 2021;9:46.

    Article  CAS  Google Scholar 

  2. Xu X. Imagery non-photorealistic art processing in modern product design relying on an inline block optimization hierarchy tree algorithm. J Image Video Proc. 2018;2018:71.

    Article  Google Scholar 

  3. de Arruda FdAPV, de Queiroz JER, Gomes HM. Non-photorealistic neural sketching. J Braz Comput Soc. 2012;18:237–50.

    Article  Google Scholar 

  4. Lizun D, Kurkiewicz T, Szczupak B. Technical examination of Liu Kang’s Paris and Shanghai painting supports (1929–1937). Herit Sci. 2021;9:37.

    Article  CAS  Google Scholar 

  5. Liu K, Gao Y, Zhang J, et al. Study on digital protection and innovative design of Qin opera costumes. Herit Sci. 2022;10:127.

    Article  Google Scholar 

  6. Liu Z, Liu K. Reproducing ancient Chinese ink depending on gelatin/chitosan and modern experimental methodology. Herit Sci. 2022;10:110.

    Article  Google Scholar 

  7. Chi MT, Lee TY. Stylized and abstract painterly rendering system using a multiscale segmented sphere hierarchy. IEEE Trans Visual Comput Graphics. 2005;12:61–72.

    Google Scholar 

  8. Ying-Ling HM. Introducing the art of modern China: trends in exhibiting modern Chinese painting in Britain, c 1930–1980. J Hist Collect. 2019;31:383–401.

    Article  Google Scholar 

  9. Li J, Shen Y, Lei L. Chinese Ink Stylized Drawing for Digital Images. 2018 5th International Conference on Systems and Informatics (ICSAI). IEEE. 2018. p. 644-648

  10. Lv C, Li Z, Shen Y, et al. SeparaFill: Two generators connected mural image restoration based on generative adversarial network with skip connect. Herit Sci. 2022;10:135.

    Article  Google Scholar 

  11. Umenhoffer T, Szirmay-Kalos L, Szécsi L, et al. An image-based method for animated stroke rendering. Vis Comput. 2018;34:817–27.

    Article  Google Scholar 

  12. Yan M, Li W, Chan CA, et al. PECS: towards personalized edge caching for future service-centric networks. China Commun. 2019;16:93–106.

    Article  Google Scholar 

  13. Yu JH, Luo GM, Peng QS. Image-based synthesis of Chinese landscape painting. J Comput Sci Technol. 2003;18:22–8.

    Article  Google Scholar 

  14. Wang CM, Wang RJ. Image-based color ink diffusion rendering. IEEE Trans Visual Comput Graphics. 2007;13:235–46.

    Article  CAS  Google Scholar 

  15. Guo F, Peng H, Tang J. A novel method of converting photograph into Chinese ink painting. IEEJ Trans Electr Electron Eng. 2015;10:320–9.

    Article  Google Scholar 

  16. Chen S. Exploration of artistic creation of Chinese ink style painting based on deep learning framework and convolutional neural network model. Soft Comput. 2020;24:7873–84.

    Article  Google Scholar 

  17. Kazi RH, Igarashi T, Zhao S, et al. Vignette: interactive texture design and manipulation with freeform gestures for pen-and-ink illustration. Proc SIGCHI Conf Hum Factor Comput Syst. 2012. https://doi.org/10.1145/2207676.2208302.

    Article  Google Scholar 

  18. Yang IS, Yu YJ, Lee DH. Ink-and-wash painting based on the image of pine tree using mean curvature flow. Proc 11th ACM SIGGRAPH Int Conf Virtual-Reality Continuum Appl Industry. 2012. https://doi.org/10.1145/2407516.2407564.

    Article  Google Scholar 

  19. Chung CY, Huang SH. Interactively transforming Chinese ink paintings into realistic images using a border enhance generative adversarial network. Multimed Tools Appl. 2022. https://doi.org/10.1007/s11042-022-13684-4.

    Article  Google Scholar 

  20. Dong L, Lu S, Jin X. Real-time image-based Chinese ink painting rendering. Multimed Tools Appl. 2014;69:605–20.

    Article  Google Scholar 

  21. Dietrich A, Gobbetti E, Yoon SE. Massive-model rendering techniques: a tutorial. IEEE Comput Graphics Appl. 2007;27:20–34.

    Article  Google Scholar 

  22. Suarez J, Belhadj F, Boyer V. Real-time 3D rendering with hatching. Vis Comput. 2017;33:1319–34.

    Article  Google Scholar 

  23. Wang J, Shen Y, Ding W. A Method for Ink-Wash Painting Rendering for 3D Scenes. 2018 International Joint Conference on Information, Media and Engineering (ICIME). IEEE. 2018. p. 205-210.

  24. Li F, Li S. real-time rendering of 3D animal models in Chinese ink painting style. 2020 International Conference on Culture-oriented Science & Technology (ICCST). IEEE. 2020. p. 284-287.

  25. Sun M, Tian T, Sun J. Artwork-based 3D ink style modeling and rendering. 2009 Sixth International Conference on Computer Graphics, Imaging and Visualization. IEEE. 2009. p.89-95.

  26. Huang L, Hou Z, Zhao Y, et al. Research progress on and prospects for virtual brush modeling in digital calligraphy and painting. Front Info Technol Electronic Eng. 2019;20:1307–21.

    Article  Google Scholar 

  27. Penner E, Borshukov G. Pre-integrated skin shading. Gpu Pro. 2011;2:41–55.

    Google Scholar 

  28. Saboret L, Alliez P, Lévy B. CGAL 4.9.1 - Planar Parameterization of Triangulated Surface Meshes. https://doc.cgal.org/4.9.1/Surface_mesh_parameterization/index.html. Accessed 6 Nov 2022.

  29. Tutte WT. How to draw a graph. Proc Lond Math Soc. 1963;1:743–67.

    Article  Google Scholar 

  30. Floater MS. Mean value coordinates. Comput Aided Geometric Design. 2003;20:19–27.

    Article  Google Scholar 

  31. Liu L, Zhang L, Xu Y, et al. A local/global approach to mesh parameterization. Computer Graphics Forum. 2008;27:1495–504.

    Article  Google Scholar 

  32. Ruales J I. A Partial Implementation in Matlab of Kwatra et al.'s paper Graphcut Textures: Image and Video Synthesis Using Graph Cuts. https://github.com/jruales/Graphcut-Textures. Accessed 6 Nov 2022.

  33. Li D, Bai C. An efficient rendering method of wash-ink strokes and its applications. J Computer-Aided Design Comput Graph. 2014;26:356–63.

    Google Scholar 

  34. Lagae A, Lefebvre S, Drettakis G, et al. Procedural noise using sparse Gabor convolution. ACM Transactions Graphics. 2009;54:1–10.

    Article  Google Scholar 

Download references

Acknowledgements

None.

Funding

This paper was supported in part by the Open Project of Key Laboratory of Audio and Video Repair and Evaluation, Ministry of Culture and Tourism (Grant No. 2021KFKT007), and the Fundamental Research Funds for the Central Universities (Grant No. CUC220B009).

Author information

Authors and Affiliations

Authors

Contributions

All the authors contributed to the current work. YM and WJ devised the study plan and led the writing of the paper. SYH and WJ arranged the data of experiment. YM and LCH supervised the entire process and provided constructive advice. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Ming Yan.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, M., Wang, J., Shen, Y. et al. A non-photorealistic rendering method based on Chinese ink and wash painting style for 3D mountain models. Herit Sci 10, 186 (2022). https://doi.org/10.1186/s40494-022-00825-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40494-022-00825-z

Keywords