使用java通过ApacheIgnite访问igfs文件

fkaflof6  于 2021-06-02  发布在  Hadoop
关注(0)|答案(2)|浏览(256)

我是ApacheIgnite的新手,需要通过java访问igfs文件。到目前为止,我尝试的是:

import java.io.InputStream;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteFileSystem;
import org.apache.ignite.Ignition;
import org.apache.ignite.igfs.IgfsPath;

public class Ignite_Spark {

    public static void main(String[] args) throws Exception {
        Ignite ignite = Ignition.start("/usr/local/hadoop/apache-ignite-hadoop-1.6.0-bin/config/default-config.xml");
        Ignite ignites = Ignition.ignite();

        IgniteFileSystem fs = ignites.fileSystem("igfs");

        IgfsPath workDir = new IgfsPath("igfs://localhost:8020/ignite/fs");
        fs.mkdirs(workDir);

        IgfsPath filePath = new IgfsPath(workDir, "ttypes.csv");

        try (InputStream in = fs.open(filePath)) {
            in.read();
        }
    }

}

但当我运行这个程序时,我得到了如下异常:
原因:class org.apache.ignite.ignitecheckedexception:未能示例化SpringXML应用程序上下文(确保spring配置中使用的所有类都存在于类路径中)[springurl=file:/usr/local/hadoop/ignite/config/default config.xml]

oipij1gg

oipij1gg1#

Data inserting and retriving from igfs file system

    My csc File(test.csv)
    ////////////////////////////////
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    eke,lelel,elelel,lelel
    w2,2,,2,2,,2,2,2,
    /////////////////////////////////

    Here configuration file is "D:/apache-ignite-fabric-1.5.0-b1-bin/examples/config/filesystem/example-igfs.xml"

    java code:
    //////////////////////////////////////////////////////////////////////////////////////start////////////////////////////////////////
    package org.apache.ignite.igfs;
    //package org.apache.ignite.igfs;
    import java.io.FileReader;

    import java.io.BufferedReader;
    import java.io.IOException;
    import java.io.InputStream;
    import java.io.InputStreamReader;
    import java.io.OutputStream;
    import org.apache.ignite.Ignite;
    import org.apache.ignite.Ignition;
    import org.apache.ignite.igfs.IgfsPath;

    public class test1 {

    public static void main(String[] args) throws IOException {
           // Here config file example-igfs.xml contanis igfs configuration
           Ignite ignite = Ignition.start("D:/apache-ignite-fabric-1.5.0-b1-bin/examples/config/filesystem/example-igfs.xml");
           org.apache.ignite.IgniteFileSystem fs = ignite.fileSystem("igfs");

           //creating a directory in igfs
           IgfsPath dir = new IgfsPath("/workDir");

           //creating a file 
           IgfsPath file = new IgfsPath(dir, "myFile1");

           //deleting files and directory to avoid file overwrite
           fs.delete(dir,true);
           fs.mkdirs(dir);

           //reading data from desktop("C:\\Users\\rb059\\Desktop\\test.csv") and writing into igfs file("MyFile1")

           try (OutputStream out = fs.create(file, true)) {
               String csvFile = "C:\\Users\\rb059\\Desktop\\test.csv";
               String line = "";
               String cvsSplitBy = ",";
               BufferedReader br = new BufferedReader(new FileReader(csvFile));

                    while ((line = br.readLine()) != null) {

                                    String s = line + "\n";

                                    out.write(s.getBytes());
                                    System.out.println("inserting data from" + csvFile + "    to   " + file +"  inserted record is " + s);

                    }

                }

           //getting data from ignite file 
           try (InputStream in = fs.open(file)) {

                    System.out.println("this is data retriving from" + file);
                    String line = "";
                    BufferedReader br = new BufferedReader(new InputStreamReader(in));

                           while ((line = br.readLine()) != null) {
                               System.out.println(line);
                           }

                }

       }

    }
    ///////////////////////////////////////////////java code end/////////////////////////////////////////

    output:

    [11:34:19]   /  _/ ___/ |/ /  _/_  __/ __/ 
    [11:34:19]  _/ // (7 7    // /  / / / _/   
    [11:34:19] /___/\___/_/|_/___/ /_/ /___/  
    [11:34:19] 
    [11:34:19] ver. 1.5.0-b1#20151202-sha1:062d440c
    [11:34:19] 2015 Copyright(C) Apache Software Foundation
    [11:34:19] 
    [11:34:19] Ignite documentation: http://ignite.apache.org
    [11:34:19] 
    [11:34:19] Quiet mode.
    [11:34:19]   ^-- Logging to file 'D:\apache-ignite-fabric-1.5.0-b1-bin\work\log\ignite-b474b4d7.0.log'
    [11:34:19]   ^-- To see**FULL**console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
    [11:34:19] 
    [11:34:19] OS: Windows 7 6.1 amd64
    [11:34:19] VM information: Java(TM) SE Runtime Environment 1.8.0_111-b14 Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.111-b14
    [11:34:19] Initial heap size is 128MB (should be no less than 512MB, use -Xms512m -Xmx512m).
    [11:34:21] Configured plugins:
    [11:34:21]   ^-- None
    [11:34:21] 
    [11:34:21] Security status [authentication=off, tls/ssl=off]
    [11:34:26] Attempting to start more nodes than physical RAM available on current host (this can cause significant slowdown)
    [11:34:30] Performance suggestions for grid  (fix if possible)
    [11:34:30] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
    [11:34:30]   ^-- Disable peer class loading (set 'peerClassLoadingEnabled' to false)
    [11:34:30]   ^-- Use default binary marshaller (do not set 'marshaller' explicitly)
    [11:34:30] 
    [11:34:30] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
    [11:34:30] 
    [11:34:30] Ignite node started OK (id=b474b4d7)
    [11:34:30] Topology snapshot [ver=18, servers=18, clients=0, CPUs=4, heap=31.0GB]
    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is 123,111,wqw,131,11221,1212

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is eke,lelel,elelel,lelel

    inserting data fromC:\Users\rb059\Desktop\test.csv    to   /workDir/myFile1  inserted record is w2,2,,2,2,,2,2,2,

    this is data retriving from/workDir/myFile1
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    123,111,wqw,131,11221,1212
    eke,lelel,elelel,lelel
    w2,2,,2,2,,2,2,2,
    [11:34:31] New version is available at ignite.apache.org: 1.7.0
qncylg1j

qncylg1j2#

我相信您可能需要包含IgniteSpringMVN依赖项,以便使用spring.xml文件启动ignite。

相关问题